ptg16105617
From the Library of YIGUANG HU
ptg16105617
The Go
Programming
Language
From the Library of YIGUANG HU
ptg16105617
This page intentionally left blank
From the Library of YIGUANG HU
ptg16105617
The Go
Programming
Language
Alan A. A. Donovan
Google Inc.
Brian W. Kernighan
Princeton University
New York • Boston • Indianapolis • San Francisco
Toronto • Montreal • London • Munich • Paris • Madrid
Capetown • Sydney • Tokyo • Singapore • Mexico City
From the Library of YIGUANG HU
ptg16105617
Many of the designations used by manufacturers and sellers to distinguish their products are claimed
as trademarks. Where those designations appear in this book, and the publisher was aware of a trade-
mark claim, the designations have been printed with initial capital letters or in all capitals.
e authors and publisher have taken care in the preparation of this book, but make no expressed
or implied warranty of any kind and assume no responsibility for errors or omissions. No liability is
assumed for incidental or consequential damages in connection with or arising out of the use of the
information or programs contained herein.
For information about buying this title in bulk quantities, or for special sales opportunities (which
may include electronic versions; custom cover designs; and content particular to your business, train-
ing goals, marketing focus, or branding interests), please contact our corporate sales department at
corpsales@pearsoned.com or (800) 382-3419.
For government sales inquiries, please contact governmentsales@pearsoned.com.
For questions about sales outside the United States, please contact international@pearsoned.com.
Visit us on the Web: informit.com/aw
Library of Congress Control Number: 2015950709
Copyright © 2016 Alan A. A. Donovan & Brian W. Kernighan
All rights reserved. Printed in the United States of America. is publication is protected by copyright,
and permission must be obtained from the publisher prior to any prohibited reproduction, storage in a
retrieval system, or transmission in any form or by any means, electronic, mechanical, photocopying,
recording, or likewise. To obtain permission to use material from this work, please submit a written
request to Pearson Education, Inc., Permissions Department, 200 Old Tappan Road, Old Tappan, New
Jersey 07675, or you may fax your request to (201) 236-3290.
Front cover: Millau Viaduct, Tarn valley, southern France. A paragon of simplicity in modern engi-
neering design, the viaduct replaced a convoluted path from capital to coast with a direct route over
the clouds. © Jean-Pierre Lescourret/Corbis.
Back cover: the original Go gopher. © 2009 Renée French. Used under Creative Commons Attribu-
tions 3.0 license.
Typeset by the authors in Minion Pro, Lato, and Consolas, using Go, gro , ghostscript, and a host of
other open-source Unix tools. Figures were created in Google Drawings.
ISBN-13: 978-0-13-419044-0
ISBN-10: 0-13-419044-0
Text printed in the United States on recycled paper at RR Donnelley in Crawfordsville, Indiana.
First printing, October 2015
From the Library of YIGUANG HU
ptg16105617
Fo r Leil a an d Me g
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
This page intentionally left blank
From the Library of YIGUANG HU
ptg16105617
Cont ents
Pr eface xi
TheOrigins ofGoxii
TheGoPro jec t xiii
Organizat ionofthe Bookxv
Wh ere toFindMoreInfor mat ionxvi
Ac knowledgments xvii
1. Tutorial1
1.1. Hel lo, Wor ld1
1.2. Command-L ineArguments 4
1.3. FindingDup lic ateLines 8
1.4. Animated GIFs 13
1.5. FetchingaURL 15
1.6. FetchingURLs Con cur rently17
1.7. A We b Server 19
1.8. Loose End s 23
2. Pro gramStr ucture27
2.1. Names 27
2.2. Declarat ions 28
2.3. Var iables 30
2.4. Assig nments 36
2.5. Typ e Decl arat ions 39
2.6. Packages andFiles 41
2.7. Scope 45
vii
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
viii CONTENTS
3. Basic Data Typ es51
3.1. Int egers 51
3.2. Float ing-Point Numbers 56
3.3. Complex Numbers 61
3.4. Boole ans63
3.5. Str ings64
3.6. Con stants75
4. Com positeTyp es81
4.1. Arrays 81
4.2. Slices 84
4.3. Maps 93
4.4. Str ucts99
4.5. JSON107
4.6. Text andHTMLTempl ates113
5. Func tions 119
5.1. FunctionDeclarat ions 119
5.2. Rec ursion121
5.3. MultipleRetur n Va lues 124
5.4. Erro rs127
5.5. FunctionValues 132
5.6. Anony mou s Func tions 135
5.7. Var iadic Functions 142
5.8. Defer red FunctionCal ls143
5.9. Panic 148
5.10. Recov er151
6. Metho ds 155
6.1. Met hod Declarat ions 155
6.2. Met hodswit h aPoint erReceiver158
6.3. ComposingTyp es by Str uct Emb edding161
6.4. Met hod Values andExpressions 164
6.5. Example: Bit Vec tor Typ e 165
6.6. Encapsulat ion168
7. Interfaces171
7.1. Int erfaces as Contrac ts 171
7.2. Int erface Typ es 174
7.3. Int erface Satisfac tion175
7.4. ParsingFlags wit h flag.Value 179
7.5. Int erface Values 181
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
CONTENTSix
7.6. Sor tingwit h sort.Interface 186
7.7. The http.Handler Interface 191
7.8. The error Interface 196
7.9. Example: ExpressionEvaluator197
7.10. Typ e As ser tions 205
7.11. Discriminat ingError s with Typ e As ser tions 206
7.12. Quer yingBeh avior s with Int erface Typ e As ser tions 208
7.13. Typ e Sw itch es210
7.14. Example: Token-B ased XML Decoding213
7.15. A Fe w Wo rds ofAdv ice 216
8. Gor o utines and Channels 217
8.1. Goroutines217
8.2. Example: Con cur rentClo ckSer ver 219
8.3. Example: Con cur rentEch o Server 222
8.4. Channel s 225
8.5. Looping inParal lel234
8.6. Example: Con cur rentWeb Craw ler 239
8.7. Multiplexingwit h select 244
8.8. Example: Con cur rentDirec tor y Traversal247
8.9. Cancellat ion251
8.10. Example: ChatSer ver 253
9. Concurrency withShared Vari ables257
9.1. Race Con dit ion s 257
9.2. Mut ual Exc lusion: sync.Mutex 262
9.3. Read/Write Mut exes: sync.RWMutex 266
9.4. Memor y Sy nchro nizat ion267
9.5. Lazy Initializat ion: sync.Once 268
9.6. TheRace Detec tor 271
9.7. Example: Con cur rentNon-Blo cking Cache 272
9.8. Goroutinesand Threads 280
10. Pack ages and the GoTool283
10.1. Int roduc tion283
10.2. Imp ort Pat hs284
10.3. ThePackageDeclarat ion285
10.4. Imp ort Declarat ions 285
10.5. Blank Imp orts286
10.6. Packages andNaming289
10.7. TheGoTool290
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
xCONTENTS
11. Testing 301
11.1. The go test To ol302
11.2. Test Func tions 302
11.3. Cov erage318
11.4. Benchmark Func tions 321
11.5. Profiling323
11.6. Example Func tions 326
12. Reflecti on329
12.1. Why Reflec tion?329
12.2. reflect.Type and reflect.Value 330
12.3. Display,aRec ursiveValue Print er333
12.4. Example: Enco dingS-E xpressions 338
12.5. Setting Var iables wit h reflect.Value 341
12.6. Example: DecodingS-E xpressions 344
12.7. AccessingStr uct Field Tags 348
12.8. Displaying the Met hodsofaTyp e 351
12.9. A Wo rdofCaution 352
13. Low-L evel Pro gramming353
13.1. unsafe.Sizeof, Alignof,and Offsetof 354
13.2. unsafe.Pointer 356
13.3. Example: DeepEquivalence 358
13.4. Cal lingCCodewit h cgo 361
13.5. Another WordofCaution 366
Index367
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
Pref ace
‘‘Go is anopensourc e prog rammi n g lang uagethatmak esiteasytobui ldsimpl e,rel iab le,
an d efficientsof tware.’’ (Fro m theGoweb sit e at golang.org)
Go was conceive d in Septemb er2007 byRob ert Gries emer,Rob Pike, and Ken Thomps on, all
at Google, and was announced inNov emb er2009. Thegoals ofthe langu ageand its accom-
pany ing tools weretobeexpressive, efficient in bot h comp ilation and exe cut ion,and effec tive
in writing reliableand robustprograms.
Go bears a sur face simi lar ity toC and,likeC,isatoolfor prof essionalprogrammers, achie v-
ingmaximum effe ctwit h minimum means.But it ismuchmorethananupdated versionof
C. Itbor rowsand adaptsgood ide as from manyother langu ages, whi le avoidingfeaturesthat
have led tocomplexity and unreliablecode. Its facilities for con cur rency are new and efficient,
andits approach to dat a abstrac tionand obj e ct-oriente d prog rammingisunu sually flexible. It
hasaut omat ic memory managementor garb age col lec tion.
Go isesp eci ally wel l suit edfor bui ldinginf rastr ucturelikenet wor ked ser vers, andtools and
systems for prog rammers, but it istruly a general-pur pos e language and nd s us e in domains
as divers e as graphics, mobileapp lic ations,and machinelearning. Ithas becom e popu lar as a
repl acementfor unt ypedscr ipt ing langu ages because itbal ances expressivenesswit h safety :
Go programstypic ally run fasterthanprogramswritt enindynamic langu ages andsuf fer far
fe wer crashesdue tounexp ected typ e er ror s.
Go isanopen-s ource pro jec t,sosourcecodefor itscompi ler,librar ies, andtools is fre ely avai l-
able toany one.Contr ibution s to the pro jec t come fro m an active worldw ide community.Go
runs onUnix-li kesystemsLinux, Fre eBSD, OpenBSD, Mac OS X—andonPlan9and
Micros oft Windows. Programswritt eninone ofthese env iro nmentsgeneral lywor k without
mo dification on the others.
xi
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
xii PREFACE
Thisbookismeant tohelpyou start usingGoeffec tive lyrig htaway andtouse itwel l,tak ing
full advantage ofGoslangu agefeaturesand stand ard librar ies to write clear, idiomatic, and
efficientprograms.
TheOrigins ofGo
Like biolog ical species, successf ullangu ages beget offspring thatincor poratethe advantagesof
their ancestors;int erbre e dingsom etimesleads tosur prisingstrengt hs; and, ver y occasionally,
aradic al ne w fe ature ariseswit houtpre cedent. Wecan lear n alot about why a langu ageisthe
way itisand whatenv iro nment ithas beenadapt edfor bylooking atthese influences.
The figurebelow shows the most imp ortantinfluences ofearlier programminglangu ages on
thedesig n of Go.
Go issom etimesdes crib edasa‘‘C-li kelangu age,’’ or as ‘‘Cfor the 21stcentury.’’ Fr omC,Go
in her ite d itsexpressionsyntax, cont rol-flow statements, basic dat a types, cal l-by-value param-
eter passing, point ers,and above all,Csemp hasisonprogramsthatcompi letoefficient
machinecodeand cooperatenatural lywit h theabstrac tions ofcur rentoperat ingsystems.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
THE ORIGINS OF GO xiii
Butthere are other ancestors inGosfami lytre e.One maj orstreamofinfluence comesfro m
languagesbyNik lausWir th, beg inningwit h Pascal.Modula-2 inspired the packagecon cept.
Ob ero n eliminated the distinc tionbet weenmoduleint erface files andmoduleimp lementation
files. Obero n-2 influence d thesyntaxfor packages, imports, anddeclarat ions,and Obj e ct
Ob ero n prov ide d thesyntaxfor met hod declarat ions.
Anot her lineage among Gosancestors,and one thatmakes Godistinc tiveamong recent
prog ramminglangu ages, isa sequence oflit tle-k now n research langu ages deve lop edatBel l
Labs, allins pired bythe con ceptof commu nicating sequent ial pro cesses (CSP) fro m To ny
Ho aresseminal 1978 pap er on the found ation s of con cur rency.InCSP,aprogram isaparal lel
comp osition of processesthathavenoshare d st ate;the pro cessescommunic ateand synchro-
nize usingchannel s.But HoaresCSP was a for mal langu agefor des cribingthe fundament al
concepts ofcon cur rency,not a programminglangu agefor writing exe cut ableprograms.
RobPikeand othersbegan toexp erimentwit h CSPimp lementation s as actu allangu ages. The
rs t was cal le d Sque ak (‘‘Alangu agefor communic atingwit h mice’’), whichprovide d alan-
gu agefor handlingmou seand key board events, wit h st aticallycre ate d ch annel s.Thiswas
fo llowe d by Newsque ak, whichoffered C-li kestatement and expressionsyntaxand Pas cal-li ke
type not ation.Itwas a purelyfunctionallangu agewit h garb agecol lec tion,again aimed at
managing key board , mous e,and windowevents. Channel s became rs t-class values, dynami-
callycre ate d andstorable in variables.
ThePlan9operat ingsystemcar r iedthese ide as forwardinalangu agecal le d Alef.Alef tried
to makeNewsque ak aviablesystemprogramminglangu age, but itsomissionofgarb agecol-
le ction made conc urrency too painf ul.
Ot her cons tructions inGoshowthe influence ofnon-ancestral genes hereand there;for
example iota is loosely fro m APL, andlexic al scop e with neste d func tions isfro m Scheme
(andmostlangu ages since). Heretoo wefind nove l mu tat ions.Gosinnovat ive slices provide
dy namic arrays wit h efficientrandomaccessbut als o permit sop histicate d sh aring
ar rangementsreminiscentoflin ked lists. And the defer st atement isnew wit h Go.
TheGoProject
Al l prog ramminglangu ages reflec t theprogrammingphi losop hyoftheir creators,whichoften
includes a significant component ofreactiontothe perceive d shortcomings ofearlier lan-
gu ages. TheGopro jec t was bor neoffrust rat ionwit h several sof twaresystems atGooglethat
were suf fer ingfro m an explosionofcomplexity.(Thispro blem isbynomeans unique to
Go ogle.)
As Rob Pikeput it, ‘‘comp lexity ismultiplic ative’’: fixingapro blem bymak ingone par t of the
systemmorecomplex slowlybut surelyaddscomplexity toother par ts. Wi thcon stant pres-
sure toadd featuresand opt ion s andconfigurat ions,and toshipcodequickly, itseasy to
neglec t simplicity,eventhoug h in the lon g runsimplicity isthe key togood sof tware.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
xiv PREFACE
Simp licity requires morewor k at the beg inningofapro jec t to reduce an ideatoits essenceand
more dis cip lineoverthe lifet imeofapro jec t to distinguish good changesfro m badorper ni-
cious ones. Wit h sufficienteffor t,agood change can beaccommodated wit houtcompromis-
ingwhatFre d Brooks cal le d the ‘‘conceptu alint egr ity’’ of the desig n butabad change cannot,
andaper nicious change trades simplicity for itsshallow cou sin, conv enience.Onlythrough
simplicity ofdesig n canasystemremain stable, sec ure , andcoh erent asitgrows.
TheGopro jec t includes the langu ageits elf,its tools and stand ard librar ies, andlastbut not
le ast, a cultural agenda ofradic al simplicity.Asarecenthig h-leve l language , Go has the bene-
tofhindsig ht, andthe basics aredon e we ll: ithas garbagecol lec tion,apackagesystem, rs t-
cl ass functions,lexic al scop e,asystemcal l interface,and immut ablestr ingsinwhichtext is
general lyencoded inUTF-8. But it has comp arat ive lyfew featuresand isunlikelytoadd
more . Fo r inst ance, ithas noimp licitnumer ic conv ersions,nocon str uctor s or destr uctor s,no
op erator overloading, nodefau ltparameter values, noinher itance, nogener ics, no exception s,
no macros, nofunctionannot ation s,and no thread-lo cal storage. The langu ageismatureand
st able, and guarante esbackwards compatibi lit y:older Goprogramscan becompi led and run
with newer versions ofcompi lersand stand ard librar ies.
Go has enough ofatyp e systemtoavoid mostofthe carelessmistakesthatplague program-
mers indynamic langu ages, but it has a simpler typ e systemthancomparable typ edlangu ages.
Thisappro ach can som etimesleadtoisolate d pockets of ‘‘untyped’’ prog rammingwit hin a
broader framewor k of typ es, andGoprogrammersdonot gotothe lengt hsthatC++ or
Haskel l prog rammersdotoexpress safet y prop erties as typ e-bas edpro ofs. But inprac tice Go
givesprogrammersmuchofthe safet y andrun-t imeper for mance benefits ofarel ative ly
st ron g type systemwit houtthe burden ofacomplex one.
Go encourages an awarenessofcontemp orar y comp utersystemdesig n, partic ularlythe
importance oflocality.Its bui lt-in dat a typesand most librar y data str uctures arecraf ted to
work natural lywit houtexplicitinitializat ionorimp licitcon str uctor s,sorel ative lyfew mem-
or y al location s andmemor y wr itesare hidden in the code. Gosaggregatetyp es (str uctsand
ar rays) hold their elements direc tly,requir inglessstorageand fewer allo cat ions and point er
indirec tion s than langu ages thatuse indirec t elds.And since the moder n comp uterisapar-
al lelmachine, Gohas conc urrency featuresbas edonCSP,asmention edearlier.The var iable-
size stacksofGoslig htweig htthreads or goro utines areinitial lysmall enoug h that creating one
goro utine ische apand cre ating a millionisprac tic al.
Gosstand ard librar y,often descr ibedascomingwit h ‘‘batt eries include d,’’ prov ides cle an
buildingblo cks andAPIsfor I/O,text pro cessing, graphics, cryptography,net wor king, and
dist ribut edapp lic ations,wit h supp ort for manystand ard file for mats andpro tocol s.The
librar ies andtools make extensive use ofconvent ion toreduce the need for configurat ionand
explanation,thu s simplif yingprogram logic andmak ingdiverse Go programsmoresimi lar to
each other andthu s easier tolearn.Pro jec tsbui ltusingthe go to oluse onlyfile andidentifier
namesand anocc asionalspeci al commenttodeter mineall the librar ies, exec utables, tests,
benchmarks, examples, platfor m-specificvar iants, anddocumentation for a proj e ct; the Go
source its elf cont ainsthe bui ld sp ecification.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
THE GO PROJECTxv
Or ganization ofthe Book
We assume thatyou haveprogrammed inone ormoreother langu ages, whether comp ile d li ke
C, C++, andJava, orint erprete d li kePyt hon,Ruby, and JavaS crip t,sowewontspell out every-
thing as if for a tot albeg inner.Sur face sy ntaxwill befami liar, as will var iables andcon stants,
expressions,control ow,and functions.
Chapter1 isatut orialonthe basic cons tructsofGo, int roduce d thro ugh a dozen programsfor
ever ydaytasks like readingand writing files, for matting text, creating images, andcommuni-
cating wit h Internet clients andser vers.
Chapter2 des crib esthe str uctural elements ofaGoprogram—de clarat ions,var iables, new
types, packages and files, andscope.Chapt er3 dis cussesnumbers,boole ans, str ings, andcon-
st ants, andexplainshow topro cessUnico de.Chapt er4 des crib escomposite typ es, that is,
typesbui ltupfro m simpler onesusingarrays, maps, str ucts, and sli ces,Gosappro ach to
dy namic lists. Chapt er5 cov ers functions and dis cusseserror handling, panic and recover,
andthe defer st atement.
Chapters 1 through 5 are thu s thebasics, things thatare par t of any mainst reamimp erat ive
language . Gossyntaxand sty lesom etimesdif fer fro m ot her langu ages, but mostprogram-
mers will pickthemupquickly. The remainingchapt ers focus ontopics where Gosappro ach
is lessconvent ion al: met hods, interfaces, conc urrency,packages, testing , andreflec tion.
Go has an unusualappro ach to obj e ct-oriente d prog ramming. There are noclass hierarchies,
or indeed any class es; comp lex obj e ctbeh avior s arecre ate d from simpler onesbycomposition,
notinher itance. Met hodsmay beass oci ated wit h anyuser-define d type,not juststr uctures,
andthe rel ation shipbet weencon crete typ es andabstrac t types(interfaces)isimp licit, soa
concrete typ e maysat isf y an interface thatthe typ esdesig ner was unawareof. Met hodsare
covere d in Chapt er6and int erfaces in Chapt er7.
Chapter8 presentsGosappro ach to con cur rency,whichisbas edonthe ide a of communic at-
ingsequential pro cesses(CSP), embodie d by goroutinesand channel s.Chapt er9 explainsthe
more tradition alasp ectsofcon cur rency bas edonshare d var iables.
Chapter10des crib espackages, the mech anism for organizinglibrar ies. Thischapt erals o
shows how tomakeeffec tive use ofthe go to ol, whichprovides for compi lat ion, testing ,
benchmarking , prog ram formatting , do cumentation,and manyother tasks, allwit hin a single
command.
Chapter11deals wit h test ing , whereGotakes a not ably lig htweig htappro ach,avoiding
abstrac tion-l aden framewor ksinfavor of simplelibrar ies andtools.The testing librar ies
prov ide a found ation atopwhichmorecomplex abstrac tions can bebui lt if necessary.
Chapter12dis cussesreflec tion,the abi lit y of a prog ram to examineits own represent ation
during exe cut ion.Reflec tion isapow erfultool, thoug h on e to beusedcaref ully; thischapt er
explains ndingthe rig htbal ance byshowing how itisusedtoimp lementsom e importantGo
librar ies. Chapter13explainsthe gor y det ails oflow-le vel programmingthatusesthe unsafe
packagetosteparo und Gostyp e system, andwhenthatisappro priate.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
xvi PREFACE
Each chapterhas a numberofexercis esthatyou can use totestyourunderst andingofGo, and
to explore extensions and alt ernat ivestothe examples fro m thebook.
Al l butthe most trivialcodeexamples in the bookare avai lable for dow nlo ad from the public
Gitrep ository at gopl.io.Eachexampleisidentied byits packageimp ort pat h andmay be
conv enientlyfetch ed, bui lt, andins tal le d usingthe go get command. Youll need tocho ose a
direc tor y to beyourGowor ksp ace andset the GOPATH enviro nment var iable topoint toit.
The go to olwill cre ate the direc tor y if necessary.For example:
$export GOPATH=$HOME/gobook #choose workspace directory
$goget gopl.io/ch1/helloworld #fetch, build, install
$$GOPATH/bin/helloworld #run
Hello, BF
To r un the examples, you will need atleast version1.5 ofGo.
$goversion
go version go1.5 linux/amd64
Fo llowthe ins tructions at https://golang.org/doc/install if the go to olonyourcom-
puterisolder ormissing.
WheretoFind MoreInformation
Thebestsourcefor moreinfor mat ionabout Goisthe offici al we b site, https://golang.org,
whichprovides accesstothe documentation,includingthe Go Programmi n g Lang uageSpecifi-
cati on,stand ard packages, andthe like. There are als o tutorialsonhow towrite Goand how
to write itwel l,and a wide variet y of onlinetext andvide o resources thatwill bevaluablecom-
plements tothisbook. TheGoBlog at blog.golang.org pu blishessom e of the bestwriting
on Go, wit h ar ticles onthe state ofthe langu age, plans for the fut ure , reportsonconferences,
andin-depth explanation s of a wide variet y of Go-rel ate d topics.
Oneofthe most usefulasp ectsofonlineaccesstoGo(andareg rettablelimitation of a pap er
book) isthe abi lit y to run Goprogramsfro m theweb pages thatdes crib e them. Thisfunc-
tion ality isprovide d by the GoPlayg round at play.golang.org,and may beemb edde d
within other pages, suchasthe home pageat golang.org or the documentation pages ser ved
by the godoc to ol.
ThePlayg round makes itconvenienttoper for m simpleexp eriments toche ckonesunder-
st andingofsyntax, semantics, orlibrar y packages wit h short programs, andinmanyways
takesthe place ofare ad-e val-print loop (REPL) in other langu ages. Its persistentURLs are
greatfor sharing snipp ets ofGocodewit h ot hers, for rep ortingbugs ormak ingsug gestion s.
Builtatopthe Playg round,the GoTourat tour.golang.org is a sequence ofshort int erac tive
lessons onthe basic ideas andcon str uctions ofGo, anorderly wal k thro ugh the langu age.
Theprimary shortcomingofthe Playg round and the Touristhatthe y al lowonlystand ard
librar ies to beimp orted,and manylibrar y fe atures—networ king, for example—arerestr icted
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
WHERE TOFIND MORE INFORMATION xvii
forprac tic al or sec urity reasons.The y also requireaccesstothe Int ernet tocompi leand run
each program. Sofor moreelaborateexp eriments, you will havetorun Goprogramsonyour
ow n comp uter. For tunatelythe dow nlo ad processisstraig htfor ward, soitshouldnot take
more thanafew minut estofetch the Godistr ibution fro m golang.org andstart writing and
runningGoprogramsofyourown.
SinceGoisanopen-s ource pro jec t,you can readthe codefor any typ e or function in the stan-
dard librar y on lineat https://golang.org/pkg;the samecodeispar t of the dow nlo ade d
dist ribut ion.Use thisto figureout how som ethingwor ks, ortoanswerquestion s ab out
det ails,ormerelytosee how exp erts write reallygood Go.
Acknowledgments
RobPikeand RussCox,coremembers ofthe Goteam, readthe manus cript several times wit h
greatcare; their comments oneverythingfro m word choice tooveral l st ruc tureand organiza-
tion havebeeninvaluable. Whi le prep aringthe Jap anes e transl ation,Yoshi kiShibata wentfar
beyond the cal l of duty; his met iculou s eyespott ednumerou s inconsistencies in the Eng lish
text anderror s in the code. Wegre atlyappre ciate thoro ugh revie ws andcriticalcommentson
theent ire manus cript fro m Br ian Goetz, Corey Kos ak, Ar noldRobbins, JoshBle e cherSny der,
andPeter Weinberger.
We a re indebte d to SameerAjmani,Itt aiBal aban, Dav id Crawshaw, BillyDon ohue, Jon athan
Feinberg , Andrew Gerrand, Rob ert Gries emer,JohnLinder man, Minux Ma, Bryan Mills, Bal a
Nataraj an, Cosmos Nicolaou , Paul Stanifor th, Nigel Tao,and HowardTrickey for many
helpf ulsug gestion s.Weals o than k DavidBrai lsfordand RaphLevien for typ esetting adv ice.
OureditorGregDoenchatAddison-Wesle y gotthe bal l ro llingoriginallyand has beencon-
tinuou sly helpf uleversince.The AWpro duc tionteam—Jo hnFuller,Day naIsley,Julie Nahi l,
Chut i Pras ertsith,and Barb ara Wo o dhasbeenoutst anding; author s couldnot hop e forbet-
tersup por t.
AlanDon ovan wishestothank:SameerAjmani,Chr isDemet riou, WaltDrummon d,and Reid
TatgeatGooglefor allow ing him timetowrite;Steph enDon ovan, for his adv ice andtimely
encouragement; andabove all,his wifeLei laKazemi,for her unhesitating ent husi asm and
unwaver ingsup por t forthispro jec t,despit e thelon g hoursofdistrac tionand abs ente eism
from fami lylifethatitent ailed.
Br ian Ker nighanisdeeply gratef ultofriends and col leagues for their pat ience andforbearance
as hemov edslowlyalong the pat h to underst anding, and esp eci ally tohis wifeMeg , whohas
been unfailinglysup por tiveofbook-w rit ing and somuchels e.
Ne w Yo r k
Oc tob er2015
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
This page intentionally left blank
From the Library of YIGUANG HU
ptg16105617
1
Tutorial
Thischapt erisatourofthe basic comp onentsofGo. Wehop e to provide enough infor mat ion
andexamples toget you off the gro und and doing usefulthings as quicklyaspossible. The
examples here, and indeed inthe whole book, areaimed attasks thatyou mig hthavetodoin
therealwor ld. Inthischapt erwell trytogiveyou a taste ofthe diversity ofprogramsthatone
mig htwrite inGo, ranging fro m simplefile pro cessingand a bit of graphics tocon cur rent
Internet clients andser vers. Wecer tain lywontexplain everythinginthe rs t ch apt er, but
studying suchprograms in a new langu agecan be an effec tive way toget started.
Wh enyoure learninganew langu age, theresanatural tendency towrite codeasyou wou ld
have writt enitinalangu ageyou already know. Beawareofthisbiasasyou learn Goand try
to avoid it. Weve tried toillustrateand explain how towrite good Go, souse the codehereas
aguide whenyoure writing your own.
1.1. Hello, World
Well start wit h thenow-t radition al ‘‘hel lo, wor ld’’ example, whichapp earsatthe beg inningof
TheCProgrammi n g Lang uage,publishe d in 1978. Cisone ofthe most direc t influences on
Go,and ‘‘hel lo, wor ld’’ illustrates a numberofcentral ideas.
gopl.io/ch1/helloworld
package main
import "fmt"
func main() {
fmt.Println("Hello, BF")
}
1
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
2CHAPTER 1. TUTORIAL
Go isacompi led langu age. The Gotoolchain conv ertsasourceprogram andthe things it
dep ends onint o inst ruc tions inthe nat ive machinelangu ageofacomputer. These tools are
accessedthrough a singlecommand cal le d go that has a numberofsub command s.The sim-
plestofthese sub command s is run,whichcompi les the sourcecodefro m on e or moresource
files whose names endin .go,lin ks it wit h librar ies, then runsthe resulting exe cut able file.
(Wewill use $ as the command pro mpt throughoutthe book.)
$gorun helloworld.go
No t surprisingly, thisprints
Hello, BF
Go nat ive lyhandles Unico de,soitcan pro cesstext in allthe wor ldslangu ages.
If the program ismorethanaone-shotexp eriment, itslikelythatyou wou ldwanttocompi le
it once andsavethe compi led resultfor later use.Thatisdon e with go build:
$gobuild helloworld.go
Thiscre atesanexe cut ablebinar y file cal le d helloworld that can berun anytimewit houtfur-
ther pro cessing:
$./helloworld
Hello, BF
We havelab ele d each sig nificant exampleasareminder thatyou can obt ain the codefro m the
bookssourcecoderep ository at gopl.io:
gopl.io/ch1/helloworld
If you run go get gopl.io/ch1/helloworld,itwill fetch the sourcecodeand place itinthe
correspondingdirec tor y.Theresmoreabout thistopic in Sec tion 2.6 andSec tion 10.7.
Letsnow tal k ab out the program its elf.Gocodeisorganize d into packages, whichare simi lar
to librar ies or modules in other langu ages. A packagecon sists ofone ormore .go source files
in a singledirec tor y that define whatthe packagedoes. Eachsource file beg inswit h a package
de clarat ion, here package main,thatstateswhichpackagethe file belon gsto, fol low edbyalist
of other packages thatitimp orts, andthenthe declarat ions ofthe program thatare store d in
that file.
TheGostand ard librar y hasover100 packages for commontasks like inp utand out put,
sorting, and text manipu lat ion. For ins tance,the fmt packagecontainsfunctions for printing
formatte d output and scanninginp ut. Println is one ofthe basic out put functions in fmt;it
pr intsone ormorevalues, sep arated byspaces, wit h anewlinecharac ter at the end sothatthe
values appear as a singlelineofout put.
Package main is speci al.Itdefinesastand alone exe cut ableprogram, not a librar y.Wit hin
package main the func tion main is als o sp eci al—itswhere exe cut ion of the program beg ins.
Wh ate ver main do es is whatthe program does. Of course, main wi l l normal lycal l up on func-
tion s in other packages to do muchofthe wor k,such as the function fmt.Println.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 1.1. HELLO, WORLD 3
We musttel l thecompi ler whatpackages areneeded bythissourcefile; thatsthe roleofthe
import de clarat ionthatfol lowsthe package de clarat ion. The ‘‘hel lo, wor ld’’ prog ram us es
on lyone functionfro m on e ot her package, but mostprogramswill imp ort morepackages.
Yo u mu stimp ort exac tly the packages you need.Aprog ram wi l l notcompi leifthere are
missingimp ortsorifthere are unnecessary ones. Thisstr ict requirementpre vents references
to unu sed packages fro m acc umulat ing as programsevo l ve.
The import de clarat ions mustfol low the package de clarat ion. After that, a program consists
of the declarat ions offunctions,var iables, cons tants, andtyp es (introduce d by the key words
func, var, const,and type); for the most par t, theorder ofdeclarat ions doesnot matter. This
prog ram is about asshort aspossiblesince itdeclaresonlyone function, whichintur n calls
on lyone other function. Tosavespace,wewill som etimesnot showthe package and import
de clarat ions whenpresent ing examples, but the y areinthe sourcefile andmustbethere to
comp ile the code.
Afunctiondeclarat ioncon sists ofthe key word func,the nameofthe function, a parameter
list (empt y for main), a resultlist(also emp tyhere), andthe bodyofthe function—thestate-
mentsthatdefine whatitdoes—enclos edinbraces. Well takeaclos er lo okatfunctions in
Chapter5.
Go doesnot requiresemicolons atthe end s of statementsordeclarat ions,exceptwhere two or
more app ear onthe sameline. Ineffec t,newlines fol low ing cer tain tokensare converted int o
semicolons,sowhere newlines areplace d matt ers to pro per parsingofGocode. For ins tance,
theopeningbrace { of the functionmustbeonthe samelineasthe end ofthe func de clara-
tion,not onalinebyits elf,and inthe expression x+y,anewlineisper mitt edafter but not
before the + op erator.
Go takes a stron g st anceoncodefor matting . The gofmt to olrewritescodeint o thestand ard
format, andthe go to ols fmt su bcommandapp lies gofmt to all the files in the specified pack-
age, orthe onesinthe cur rentdirec tor y by defau lt. All Gosourcefiles in the bookhavebeen
runthrough gofmt,and you shouldget into the habit of doing the samefor yourown code.
Decl aring a stand ard for mat by fiat eliminates a lotofpoint lessdeb ate about trivia and,more
importantly, enables a variet y of aut omated sourcecodetransfor mat ions thatwou ldbe
infeasible if arbit rar y formatting were allow ed.
Many text editors can beconfigured torun gofmt each timeyou saveafile,sothatyoursource
co de is always pro perly for matte d. Arel ate d to ol, goimports,addition allymanages the ins er-
tion and removal ofimp ort declarat ions asneeded.Itisnot par t of the stand ard distr ibution
butyou can obt ain itwit h this command:
$goget golang.org/x/tools/cmd/goimports
Fo r most users,the usu alway todow nlo ad andbui ld packages, run their tests, showtheir doc-
umentation,and soon, iswit h the go to ol, whichwell look at in Sec tion 10.7.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
4CHAPTER 1. TUTORIAL
1.2. Command-Line Arguments
Most programspro cesssom e inputtopro duce som e output;thatsprett y much the definition
of computing . Buthow doesaprogram get inputdat a on whichtooperate? Som e prog rams
generatetheir own data,but moreoften, inputcom esfro m an exter nal source: a file,anet wor k
connec tion,the out put of another program, a useratakey board , command-linearguments,
or the like. The next few examples will dis cusssom e of these alt ernat ives, startingwit h com-
mand-linearguments.
The os packageprovides functions and other values for dealingwit h theoperat ingsystemina
pl atfor m-indep endentfashion. Command-linearguments areavai lable toaprogram in a
var iable named Args that ispar t of the os package; thu s itsnameany where outside the os
packageis os.Args.
Thevar iable os.Args is a sli ce of str ings. Slices areafundament alnot ion inGo, and well tal k
alot moreabout themsoon. For now,thin k of a slice as a dynamic ally size d sequence s of
ar ray elements where indiv idu alelements can beaccessedas s[i] andacontiguous subse-
quence as s[m:n].The numberofelements isgiven by len(s).Asinmostother program-
minglangu ages, allindexinginGouses half-open intervalsthatinclude the rs t index but
exclude the last, because itsimplifies logic. For example, the slice s[m:n],where 0 m n
len(s),contains n-m elements.
The rs t elementof os.Args, os.Args[0],isthe nameofthe command its elf; the other ele-
mentsare the arguments thatwerepresent edtothe program whenitstarted exe cut ion.A
slice expressionofthe for m s[m:n] yields a slice thatreferstoelements m thro ugh n-1,sothe
elements weneed for our next exampleare those inthe slice os.Args[1:len(os.Args)].If m
or n is omitt ed, itdefau lts to0or len(s) resp ectively, sowecan abbreviate the desired slice as
os.Args[1:].
Heresanimp lementation of the Unix echo command, whichprintsits command-lineargu-
mentsonasingleline. Itimp ortstwo packages, whichare given as a parenthesize d list rat her
than as individu al import de clarat ions.Eit her for m is legal,but convent ion allythe listfor m is
us ed. The order ofimp ortsdoesntmatter; the gofmt to olsor tsthe packagenames into
alph abeticalorder.(Wh enthere are several versions ofanexample, wewill often number
them soyou can besureofwhichone were tal kingabout.)
gopl.io/ch1/echo1
// Echo1 prints its command-line arguments.
package main
import (
"fmt"
"os"
)
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 1.2. COMMAND-LINE ARGUMENTS5
func main() {
var s, sep string
for i := 1; i < len(os.Args); i++ {
s+=sep + os.Args[i]
sep = " "
}
fmt.Println(s)
}
Comments beg in with //.All text fro m a // to the end ofthe lineiscomment ary for
prog rammersand isignored bythe compi ler.Byconvent ion,wedes crib e each package in a
commentimmediate lypre cedingits packagedeclarat ion; for a main package, thiscomment is
on e or morecompletesentences thatdes crib e theprogram as a whole.
The var de clarat iondeclarestwo var iables s and sep,oftyp e string.Avar iable can beini-
tialize d as par t of itsdeclarat ion. Ifitisnot explicitlyinitialize d,itisimp licitlyinitialize d to
the zero value forits typ e,whichis 0 fornumer ic typesand the emp tystr ing "" forstr ings.
Thus inthisexample, the declarat ionimp licitlyinitializes s and sep to emp tystr ings. Well
have moretosay aboutvar iables anddeclarat ions inChapt er2.
Fo r numb ers,Goprovides the usu alarithmeticand log ical operator s.Whenapp lie d to
st rings, how ever, the + op erator concatenate s thevalues, sothe expression
sep + os.Args[i]
repres ents the con catenat ionofthe str ings sep and os.Args[i].The statement weusedin
theprogram,
s+=sep + os.Args[i]
is an assig nment statement that conc atenatesthe old value of s with sep and os.Args[i] and
assig nsitbackto s;itisequivalentto
s=s+sep + os.Args[i]
Theoperator += is an assig nment operator.Eacharithmeticand log ical operator like + or * has
acor respondingassig nmentoperator.
The echo prog ram couldhaveprint edits out put inaloopone pie ce at a time, but thisversion
insteadbui ldsupastr ing byrep eatedlyapp endingnew text tothe end.The str ing s st artslife
empt y,thatis, wit h value "",and eachtripthrough the loopaddssom e text toit; afterthe rs t
it erat ion, a space isals o inserted sothatwhenthe loopis finishe d,there isone space bet ween
each argument. Thisisaquadrat ic processthatcou ldbecostlyifthe numberofarguments is
large , butfor echo,thatsunlikely. Well showanumberofimp rov edversions of echo in this
ch apt erand the next thatwill dealwit h anyrealinefficiency.
Theloopindex variable i is declare d in the rs t part ofthe for lo op. The := sy mbolispar t of
a sh ort var iab ledeclarati on,astatement thatdeclaresone ormorevar iables andgives them
appropriatetyp es basedonthe initializer values; theresmoreabout this in the next chapt er.
Theincrement statement i++ adds1to i;itsequivalentto i+=1whichisintur n equivalent
to i=i+1.Theresacor respondingdecrement statement i-- that subtrac ts 1. Thes e are
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
6CHAPTER 1. TUTORIAL
st atements, not expressions asthe y areinmostlangu ages in the C fami ly, so j=i++ is illegal,
andthe y arepostfixonly, so --i is not legal either.
The for lo opisthe onlyloopstatement inGo. Ithas a numberoffor ms, one ofwhichis
illustrated here:
for
initialization
;
condition
;
post
{
// zero or more statements
}
Parent hes es arenever usedaro und the three componentsofafor lo op. The braces are
mand atory,how ever, and the openingbrace mustbeonthe sameline as the
post
st atement.
Theopt ion al
initialization
st atement isexe cut edbeforethe loopstarts. Ifitispresent,it
mu stbeasimp lestatement,thatis, a short var iable decl arat ion, an incrementorassig nment
st atement,orafunctioncal l.The
condition
is a boole an expressionthatisevaluate d at the
beginningofeachiterat ionofthe loop; if itevaluatesto true,the statementscontrol led bythe
lo opare exe cut ed. The
post
st atement isexe cut edafter the bodyofthe loop, thenthe con di-
tion isevaluate d again. Theloopend s when the con dit ion becom esfalse.
Anyofthese par ts maybeomitt ed. Ifthere isno
initialization
andno
post
,the semi-
colons may also beomitt ed:
// a traditional "while" loop
for
condition
{
// ...
}
If the con dit ion isomitt edent ire ly in any ofthese for ms, for examplein
// a traditional infinite loop
for {
// ...
}
theloopisinfinite, thoug h lo ops ofthisfor m maybeter minated insom e ot her way,likea
break or return st atement.
Anot her for m of the for lo opiterates overara nge of values fro m adat a type likeast ringora
slice.Toillustrate, heresasecon d versionof echo:
gopl.io/ch1/echo2
// Echo2 prints its command-line arguments.
package main
import (
"fmt"
"os"
)
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 1.2. COMMAND-LINE ARGUMENTS7
func main() {
s, sep := "", ""
for _, arg := range os.Args[1:] {
s+=sep + arg
sep = " "
}
fmt.Println(s)
}
In eachiterat ionofthe loop, range produces a pair ofvalues: the index andthe value ofthe
elementatthatindex. Inthisexample, wedontneed the index, but the syntaxofarange lo op
re quires thatifwedealwit h theelement, wemustdealwit h theindex too.One ide a wouldbe
to assig n theindex to an obv iou sly temporar y var iable like temp andignoreits value,but Go
do es notper mitunu sed local variables, sothiswou ldresult in a compi lat ionerror.
Thesolut ion istouse the bl ank ident ifier,whose nameis _ (t hat is, an underscore). Theblank
identifier may beusedwhene ver synt axrequires a variablenamebut prog ram log ic do es not,
forins tance todis cardanunwante d lo opindex whenwerequireonlythe elementvalue.Most
Go programmerswou ldlikelyuse range and _ to write the echo prog ram as abov e,since the
indexingover os.Args is imp licit, not explicit, andthu s easier toget rig ht.
Thisversionofthe program usesa short var iable decl arat iontodeclare and initialize s and
sep,but wecou ldequ allywel l have declare d thevar iables sep arately. There are several ways
to declare a str ing var iable; these are all equivalent:
s:=""
var s string
var s = ""
var s string = ""
Whyshouldyou preferone for m to another? The rs t form,ashort var iable decl arat ion, is
themostcompact,but it may beusedonlywit hin a function, not for package-le vel var iables.
Thesecon d form relies ondefau ltinitializat iontothe zerovalue for str ings, whichis "".The
thirdfor m is rarelyusedexceptwhendeclaring multiplevar iables. Thefourthfor m is explicit
ab out the var iablestyp e,whichisredundant whenitisthe sameasthatofthe initial value but
ne cessary inother cas es wherethe y arenot ofthe sametyp e.Inprac tice,you shouldgeneral ly
us e on e of the rs t twofor ms, wit h explicitinitializat iontosay thatthe initial value is
importantand imp licitinitializat iontosay thatthe initial value doesntmatter.
As not edabove , each timearo und the loop, the str ing s gets comp letelynew contents. The +=
st atement makes a new str ing bycon catenat ingthe old str ing , aspace charac ter,and the next
argument, thenassig nsthe newstr ing to s.The old cont entsof s arenolon g erinuse,sothe y
wi l l be garb age-collec ted indue course.
If the amountofdat a invo l vedislarge , this cou ldbecostly. A simpler andmoreefficient
solut ion wou ldbetouse the Join func tionfro m the strings package:
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
8CHAPTER 1. TUTORIAL
gopl.io/ch1/echo3
func main() {
fmt.Println(strings.Join(os.Args[1:], " "))
}
Final ly, ifwedontcareabout for mat but justwanttosee the values, perhaps for debug ging, we
canlet Println format the results for us:
fmt.Println(os.Args[1:])
Theout put of thisstatement islikewhatwewou ldget fro m strings.Join,but wit h sur-
ro undingbrackets. Any slice may beprint edthisway.
Exercis e 1.1: Mo dif y the echo prog ram to als o pr int os.Args[0],the nameofthe command
that invo ked it.
Exercis e 1.2: Mo dif y the echo prog ram to print the index andvalue ofeachofits arguments,
on e perline.
Exercis e 1.3: Experiment tomeasure the dif ference in runningtimebet weenour pot ent ial ly
inefficientversions and the one thatuses strings.Join.(Section1.6 illustrates par t of the
time package, and Sec tion 11.4 shows how towrite benchmark tests for systematicper-
formance evaluation.)
1.3. Finding DuplicateLines
Prog ramsfor file copy ing , pr int ing , searching, sor ting, count ing , andthe likeall haveasimi lar
st ruc ture: a loop overthe inp ut, som e comp utation on eachelement, andgenerat ionofout put
on the yoratthe end.Well showthree var iants ofaprogram cal le d dup;itispar tly ins pired
by the Unix uniq command, whichlooks for adj acentdup lic atelines. Thestr uctures and
packages usedare model s that can beeasi lyadapt ed.
The rs t versionof dup pr intseachlinethatapp earsmorethanonceinthe stand ard inp ut,
preceded byits count. Thisprogram introduces the if st atement,the map data typ e,and the
bufio package.
gopl.io/ch1/dup1
// Dup1 prints the text of each line that appears more than
// once in the standard input, preceded by its count.
package main
import (
"bufio"
"fmt"
"os"
)
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 1.3. FINDING DUPLICATE LINES 9
func main() {
counts := make(map[string]int)
input := bufio.NewScanner(os.Stdin)
for input.Scan() {
counts[input.Text()]++
}
// NOTE: ignoring potential errors from input.Err()
for line, n := range counts {
if n > 1 {
fmt.Printf("%d\t%s\n", n, line)
}
}
}
As wit h for,parenthes es arenever usedaro und the con dit ion inan if st atement,but braces
arerequired for the body. There can beanopt ion al else part thatisexe cut edifthe con dit ion
is false.
A map holdsaset ofkey/value pairsand provides cons tant-t imeoperat ions tostore , retr ieve,
or testfor aniteminthe set. Thekey may beofany typ e whos e values can comp are d with ==,
st rings beingthe most commonexample; the value may beofany typ e at all. Inthisexample,
thekeysare str ingsand the values are ints. Thebui lt-in function make createsanew emp ty
map; ithas other usestoo.Maps aredis cussed at lengt h in Sec tion 4.3.
Each time dup re ads a lineofinp ut, the lineisusedasakey int o themap andthe cor-
resp ondingvalue isincrement ed. The statement counts[input.Text()]++ is equivalentto
thes e twostatements:
line := input.Text()
counts[line] = counts[line] + 1
Itsnot a pro blem if the map doesntyet cont ain thatkey.The rs t time a newlineisseen, the
expression counts[line] on the rig ht-handside evaluatestothe zerovalue for itstyp e,which
is 0 for int.
To print the results, weuse another range-b ased for lo op, thistimeoverthe counts map. As
before , each iterat ionpro duces two results, a key and the value ofthe map elementfor that
ke y.The order ofmap iterat ionisnot specified,but inprac tice itisrandom, varying fro m on e
runtoanother.Thisdesig n is int ent ion al, since itpre vents programsfro m re lying onany par-
ticularorder ingwhere non e is guarante e d.
Onwardtothe bufio package, whichhelps makeinp utand out put efficientand convenient.
Oneofits mostusefulfeaturesisatyp e called Scanner that reads inp utand bre aks itint o lines
or words;itsoften the easiestway topro cessinp utthatcom esnatural ly in lines.
Theprogram usesashort var iable decl arat iontocre ate a new var iable input that referstoa
bufio.Scanner:
input := bufio.NewScanner(os.Stdin)
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
10 CHAPTER 1. TUTORIAL
Thescanner reads fro m theprogramsstand ard inp ut. Eachcal l to input.Scan() re ads the
next lineand remov esthe newlinecharac ter fro m theend;the resultcan beret rie ved bycal l-
ing input.Text().The Scan func tionretur ns true if there isalineand false when there is
no more inp ut.
Thefunction fmt.Printf,like printf in C andother langu ages, pro duces for matte d output
from a listofexpressions.Its rs t argumentisafor mat str ing thatspecifies how subsequent
arguments shouldbefor matte d. Thefor mat ofeachargumentisdeter mined byaconversion
ch arac ter,aletterfol low ing a percentsig n. Fo r example, %d formats an integeroperandusing
de cimal not ation,and %s exp ands tothe value ofastr ing operand.
Printf hasoveradozen suchconversions,whichGoprogrammerscal l verb s.Thistable isfar
from a completespecification but illustrates manyofthe featuresthatare avai lable:
%d de cimal integer
%x, %o, %b integer in hexade cimal,octal,binar y
%f, %g, %e floating-p ointnumber: 3.141593 3.141592653589793 3.141593e+00
%t boole an: true or false
%c rune (Unico de co de point)
%s st ring
%q quot edstr ing "abc" or rune 'c'
%v anyvalue in a natural for mat
%T type ofany value
%% literal percentsig n (nooperand)
Thefor mat str ing in dup1 also containsatab \t andanewline \n.Str ing lit eralsmay cont ain
such es cap e sequ ences forrepresent ing other wis e invisiblecharac ters. Printf do es notwrite a
ne wlinebydefau lt. Byconvent ion,for matting functions whose names endin f,suchas
log.Printf and fmt.Errorf,use the for matting rules of fmt.Printf,where asthose whose
namesend in ln fo llow Println,for matting their arguments as if by %v,fol low edbya
ne wline.
Many programsreadeit her fro m their stand ard inp ut, as abov e,orfro m asequence ofnamed
files. Thenext versionof dup canreadfro m thestand ard inp utorhandlealistoffile names,
using os.Open to openeachone:
gopl.io/ch1/dup2
// Dup2 prints the count and text of lines that appear more than once
// in the input. It reads from stdin or from a list of named files.
package main
import (
"bufio"
"fmt"
"os"
)
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 1.3. FINDING DUPLICATE LINES 11
func main() {
counts := make(map[string]int)
files := os.Args[1:]
if len(files) == 0 {
countLines(os.Stdin, counts)
}else {
for _, arg := range files {
f, err := os.Open(arg)
if err != nil {
fmt.Fprintf(os.Stderr, "dup2: %v\n", err)
continue
}
countLines(f, counts)
f.Close()
}
}
for line, n := range counts {
if n > 1 {
fmt.Printf("%d\t%s\n", n, line)
}
}
}
func countLines(f *os.File, counts map[string]int) {
input := bufio.NewScanner(f)
for input.Scan() {
counts[input.Text()]++
}
// NOTE: ignoring potential errors from input.Err()
}
Thefunction os.Open returnstwo values. The rs t is anopenfile (*os.File)thatisusedin
su bsequentreads bythe Scanner.
Thesecon d resu ltof os.Open is a value ofthe bui lt-in error type.If err equals the speci al
built-in value nil,the file was opene d successf ully. The file isread, and whenthe end ofthe
inputisreach ed, Close clos es thefile andreleasesany res ources. On theother hand, if err is
not nil,som ethingwentwro ng. Inthatcas e,the error value descr ibesthe pro blem. Our sim-
ple-minde d er ror handlingprintsamessage onthe stand ard error streamusing Fprintf and
theverb %v,whichdispl ays a value ofany typ e in a default for mat, and dup then car r ies on
with thenext file; the continue st atement goestothe next iterat ionofthe enclosing for lo op.
In the int erestsofkeeping codesamples toareasonablesize,our early examples areint ent ion-
al lysom ewhat cavalier abouterror handling. Cle arly wemustche ckfor anerror fro m
os.Open;how ever, weare ignor ing the lesslikelypossibi lit y that an erro r couldocc ur while
re adingthe file wit h input.Scan.Wewill not e pl aces where weve skipp ederror che cking ,
andwewill goint o thedet ails oferror handling in Sec tion 5.4.
No tice thatthe cal l to countLines precedes its decl arat ion. Functions and other package-le vel
entities may bedeclare d in anyorder.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
12 CHAPTER 1. TUTORIAL
Amap isareference to the dat a st ruc turecre ate d by make.Whenamap ispassedtoafunc-
tion,the functionreceivesa copyofthe reference,soany changesthe cal le d func tionmakes to
theunderly ing dat a st ruc turewill bevisible through the cal lersmap reference too.Inour
example, the values inserted int o the counts mapby countLines areseenby main.
Theversions of dup ab ove operateina‘‘st reaming’’ mo deinwhichinp utisreadand bro ken
into lines as needed,soinprincip lethese programscan handleanarbit rar y amount ofinp ut.
An alt ernat ive appro ach istoreadthe ent ire inp utint o memory inone big gulp,split it int o
lines allatonce, thenpro cessthe lines. Thefol low ing version, dup3,operates in thatfashion.
It int roduces the function ReadFile (f rom the io/ioutil package), whichreads the ent ire
cont entsofaname d file,and strings.Split,whichsplitsastr ing int o aslice ofsubst rings.
(Split is the opp osite of strings.Join,whichwesaw earlier.)
Weve simplified dup3 some what. First,itonlyreads name d files, not the stand ard inp ut, since
ReadFile re quires a file nameargument. Secon d,wemov edthe count ing ofthe lines back
into main,since itisnow needed inonlyone place.
gopl.io/ch1/dup3
package main
import (
"fmt"
"io/ioutil"
"os"
"strings"
)
func main() {
counts := make(map[string]int)
for _, filename := range os.Args[1:] {
data, err := ioutil.ReadFile(filename)
if err != nil {
fmt.Fprintf(os.Stderr, "dup3: %v\n", err)
continue
}
for _, line := range strings.Split(string(data), "\n") {
counts[line]++
}
}
for line, n := range counts {
if n > 1 {
fmt.Printf("%d\t%s\n", n, line)
}
}
}
ReadFile returnsabyteslice thatmustbeconverted int o a string so itcan besplit by
strings.Split.Wewill dis cussstr ingsand byteslices at lengt h in Sec tion 3.5.4.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 1.4. ANIMATED GIFS13
Un der the cov ers, bufio.Scanner, ioutil.ReadFile,and ioutil.WriteFile us e the Read
and Write methodsof *os.File,but itsrarethatmostprogrammersneed toaccessthose
lower-le vel routinesdirec tly.The hig her-le vel functions likethose fro m bufio and io/ioutil
areeasier touse.
Exercis e 1.4: Mo dif y dup2 to print the names ofall files in whicheachdup lic ated lineocc urs.
1.4. Animated GIFs
Thenext program demon strates basic usage ofGosstand ard image packages, whichwell use
to cre ate a sequence ofbit-mappedimagesand thenencodethe sequence as a GIF animat ion.
Theimages, cal le d Li ssajous gure s,wereastaplevisualeffec t in sci-fifilmsofthe 1960s. They
arethe paramet ric cur ves pro duce d by har monic osci l lat ionintwo dimensions,suchastwo
sinewaves fed int o the x and y inputs of an oscillos cope.Figure1.1 shows som e examples.
Figure 1.1. Fo urLissajous figures.
Thereare several new con str uctsinthiscode, including const de clarat ions,str uct typ es, and
comp osite lit erals. Unlikemostofour examples, thisone als o invo l vesfloating-p ointcom-
putation s.Well dis cussthese topics onlybrieflyhere, pushingmostdet ails off tolater chap-
ters,since the primary goalrig htnow istogiveyou anide a of whatGolooks like and the
kind s of things thatcan bedon e easi lywit h thelangu ageand its librar ies.
gopl.io/ch1/lissajous
// Lissajous generates GIF animations of random Lissajous figures.
package main
import (
"image"
"image/color"
"image/gif"
"io"
"math"
"math/rand"
"os"
)
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
14 CHAPTER 1. TUTORIAL
var palette = []color.Color{color.White, color.Black}
const (
whiteIndex = 0 // first color in palette
blackIndex = 1 // next color in palette
)
func main() {
lissajous(os.Stdout)
}
func lissajous(out io.Writer) {
const (
cycles = 5//number of complete x oscillator revolutions
res = 0.001 // angular resolution
size = 100 // image canvas covers [-size..+size]
nframes = 64 // number of animation frames
delay = 8//delay between frames in 10ms units
)
freq := rand.Float64() * 3.0 // relative frequency of y oscillator
anim := gif.GIF{LoopCount: nframes}
phase := 0.0 // phase difference
for i := 0; i < nframes; i++ {
rect := image.Rect(0, 0, 2*size+1, 2*size+1)
img := image.NewPaletted(rect, palette)
for t := 0.0; t < cycles*2*math.Pi; t += res {
x:=math.Sin(t)
y:=math.Sin(t*freq + phase)
img.SetColorIndex(size+int(x*size+0.5), size+int(y*size+0.5),
blackIndex)
}
phase += 0.1
anim.Delay = append(anim.Delay, delay)
anim.Image = append(anim.Image, img)
}
gif.EncodeAll(out, &anim) // NOTE: ignoring encoding errors
}
Af ter importingapackagewhose pat h hasmultiplecomponents, like image/color,werefer
to the packagewit h anamethatcom esfro m thelastcomponent.Thu s thevar iable
color.White belongstothe image/color packageand gif.GIF belongsto image/gif.
A const de clarat ion(§3.6) gives names tocon stants, thatis, values thatare xe d at comp ile
time,suchasthe numer ical parametersfor cyc les, frames, anddel ay. Like var de clarat ions,
const de clarat ions may appear at packagelevel (so the names arevisible throughoutthe pack-
age) orwit hin a function(so the names arevisible onlywit hin thatfunction). Thevalue ofa
cons tantmustbeanumber, str ing , or boole an.
Theexpressions []color.Color{...} and gif.GIF{...} are comp osite lit era ls (§4.2, §4.4.1),
acompact not ation for ins tantiat ingany ofGoscomposite typ es from a sequence ofelement
values. Here, the rs t on e is a slice andthe secon d on e is a st ruc t.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 1.5. FETCHING A URL 15
Thetyp e gif.GIF is a str uct typ e (§4.4). A st ruc t is a gro upofvalues cal le d el ds,often ofdif-
ferent typ es, that arecol lec ted toget her in a singleobj e ctthatcan betre ate d as a unit. The
var iable anim is a str uct oftyp e gif.GIF.The str uct lit eral createsastr uct value whose Loop-
Count eldisset to nframes;all other elds have the zerovalue for their typ e.The indiv idu al
elds ofastr uct can beaccessedusingdot not ation,asinthe naltwo assig nments which
explicitlyupdatethe Delay and Image elds of anim.
The lissajous func tionhas two neste d lo ops. Theout erlooprunsfor 64iterat ions,each
producingasingleframeofthe animat ion. Itcre atesanew 201&201 imagewit h apalette of
twocolor s,white and black.All pixel s areinitial lyset tothe paletteszerovalue (the zerot h
colorinthe palette), whichweset towhite. Eachpassthrough the inner loop generates a new
imagebysetting som e pixe lstoblack.The resultisapp ended,usingthe bui lt-in append func-
tion (§4.2.1), toalistofframes in anim,along wit h aspecified del ayof80ms. Final lythe
sequence offrames anddel ays isencoded int o GIF for mat andwritt entothe out put stream
out.The typ e of out is io.Writer,whichlets uswrite to a wide range ofpossibledestina-
tion s, as well showsoon.
Theinner loop runsthe two oscillator s.The x os cillator isjustthe sinefunction. The y os cil-
latorisals o asinus oid,but itsfre quency rel ative tothe x os cillator isarandomnumber
between0and 3,and its phase rel ative tothe x os cillator isinitial lyzerobut incre aseswit h
each frameofthe animat ion. Thelooprunsunt i l the x os cillator has comp leted ve full
cycles. Ateachstep, itcal ls SetColorIndex to color the pixel cor respondingto(x, y)black,
which is at position 1 inthe palette.
The main func tioncal lsthe lissajous func tion, direc ting ittowrite to the stand ard out put,
so thiscommand pro duces an animated GIF wit h frames like those inFigure1.1:
$gobuild gopl.io/ch1/lissajous
$./lissajous >out.gif
Exercis e 1.5: Change the Lissajous programscolor palette to gre enonblack,for adde d
authenticity.Tocre ate the web color #
RRGGBB
,use color.RGBA{0x
RR
,0x
GG
,0x
BB
,0xff},
whereeachpair ofhexade cimal digitsrepresentsthe int ensit y of the red,gre en, orbluecom-
ponent ofthe pixel.
Exercis e 1.6: Mo dif y theLissajous program topro duce images in multiplecolor s by adding
more values to palette andthendispl aying thembychang ing the thirdargumentof Set-
ColorIndex in som e interest ing way.
1.5. Fet ching a URL
Fo r many app lic ations,accesstoinfor mat ionfro m theInt ernet isasimp ortantasaccesstothe
lo cal file system. Goprovides a collec tion of packages, gro upedunder net,thatmakeiteasy
to sendand receive infor mat ionthrough the Int ernet, makelow-le vel networ k connec tion s,
andset upser vers, for whichGoscon cur rency features(introduce d in Chapt er8)are par tic u-
larlyuseful.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
16 CHAPTER 1. TUTORIAL
To illustratethe minimum necessary toret rie veinfor mat ionoverHTTP, h eresasimple
prog ram called fetch that fetch esthe content ofeachspecified URL andprintsitasuninter-
preted text; itsins pired bythe invaluableutilit y curl.Obv iou sly one wou ldusu allydomore
with suchdat a, butthisshows the basic idea. Wewill use thisprogram fre quentlyinthe book.
gopl.io/ch1/fetch
// Fetch prints the content found at a URL.
package main
import (
"fmt"
"io/ioutil"
"net/http"
"os"
)
func main() {
for _, url := range os.Args[1:] {
resp, err := http.Get(url)
if err != nil {
fmt.Fprintf(os.Stderr, "fetch: %v\n", err)
os.Exit(1)
}
b, err := ioutil.ReadAll(resp.Body)
resp.Body.Close()
if err != nil {
fmt.Fprintf(os.Stderr, "fetch: reading %s: %v\n", url, err)
os.Exit(1)
}
fmt.Printf("%s", b)
}
}
Thisprogram introduces functions fro m twopackages, net/http and io/ioutil.The
http.Get func tionmakes an HTTP requestand,ifthere isnoerror,retur nsthe resultinthe
resp ons e st ruc t resp.The Body eldof resp cont ainsthe ser ver respons e as a readable
st ream. Next, ioutil.ReadAll re ads the ent ire resp ons e;the resultisstore d in b.The Body
st reamisclos edtoavoid leak ingres ources, and Printf wr itesthe respons e to the stand ard
output.
$gobuild gopl.io/ch1/fetch
$./fetch http://gopl.io
<html>
<head>
<title>The Go Programming Language</title>
...
If the HTTPrequestfai ls, fetch reportsthe fai lureins tead:
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 1.6. FETCHING URLS CONCURRENTLY17
$./fetch http://bad.gopl.io
fetch: Get http://bad.gopl.io: dial tcp: lookup bad.gopl.io: no such host
In eit her erro r case, os.Exit(1) caus esthe pro cesstoexitwit h astatu s co de of 1.
Exercis e 1.7: Thefunctioncal l io.Copy(dst, src) re ads fro m src andwritesto dst.Use it
insteadof ioutil.ReadAll to copythe respons e bodyto os.Stdout withoutrequir inga
buffer large enoug h to holdthe ent ire stream. Besuretoche ckthe error resultof io.Copy.
Exercis e 1.8: Mo dif y fetch to add the prefix http:// to eachargumentURL if itismissing.
Yo u mig htwanttouse strings.HasPrefix.
Exercis e 1.9: Mo dif y fetch to als o pr int the HTTPstatu s co de,found in resp.Status.
1.6. Fet ching URLsConcurrently
Oneofthe most int erest ing and nove l aspectsofGoisits sup por t forcon cur rentprogram-
ming. Thisisalarge topic, towhichChapt er8 and Chapt er9 are devot ed, sofor now well
give youjustataste ofGosmain conc urrency mechanisms, goroutinesand channel s.
Thenext program, fetchall,doesthe samefetch ofaURLscontentsasthe pre vious example,
butitfetch esmanyURLs, allcon cur rently, sothatthe pro cesswill takenolon g erthanthe
longest fetch rat her thanthe sum ofall the fetch times. Thisversionof fetchall discards the
resp ons esbut rep ortsthe size andelaps edtimefor eachone:
gopl.io/ch1/fetchall
// Fetchall fetches URLs in parallel and reports their times and sizes.
package main
import (
"fmt"
"io"
"io/ioutil"
"net/http"
"os"
"time"
)
func main() {
start := time.Now()
ch := make(chan string)
for _, url := range os.Args[1:] {
go fetch(url, ch) // start a goroutine
}
for range os.Args[1:] {
fmt.Println(<-ch) // receive from channel ch
}
fmt.Printf("%.2fs elapsed\n", time.Since(start).Seconds())
}
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
18 CHAPTER 1. TUTORIAL
func fetch(url string, ch chan<- string) {
start := time.Now()
resp, err := http.Get(url)
if err != nil {
ch <- fmt.Sprint(err) // send to channel ch
return
}
nbytes, err := io.Copy(ioutil.Discard, resp.Body)
resp.Body.Close() // don'tleak resources
if err != nil {
ch <- fmt.Sprintf("while reading %s: %v", url, err)
return
}
secs := time.Since(start).Seconds()
ch <- fmt.Sprintf("%.2fs %7d %s", secs, nbytes, url)
}
Heres an example:
$gobuild gopl.io/ch1/fetchall
$./fetchall https://golang.org http://gopl.io https://godoc.org
0.14s 6852 https://godoc.org
0.16s 7261 https://golang.org
0.48s 2475 http://gopl.io
0.48s elapsed
A goro utine is a con cur rentfunctionexe cut ion.Ach ann el is a communic ationmechanism
that allows one goroutine topassvalues ofaspecified typ e to another goroutine.The function
main runs inagoroutine and the go st atement cre atesaddition algoroutines.
The main func tioncre atesachannel ofstr ingsusing make.For eachcommand-lineargument,
the go st atement inthe rs t range loopstartsanew goroutine thatcal ls fetch asy nchro nou sly
to fetch the URL using http.Get.The io.Copy func tionreads the bodyofthe respons e and
discards itbywriting tothe ioutil.Discard output stream. Copy returnsthe bytecount,
along wit h anyerror thatocc urre d.Aseachresultarrives, fetch send s asummar y lineonthe
ch annel ch.The secon d range loopin main re ceivesand printsthose lines.
Wh enone goroutine att emp tsasendorreceive onachannel,itblo cks until another goroutine
attemp tsthe cor respondingreceive orsendoperat ion, at whichpoint the value istransfer red
andbot h goro utinespro ceed.Inthisexample, each fetch send s avalue (ch <- expre ssi on)on
thechannel ch,and main re ceivesall ofthem(<-ch). Hav ing main do allthe print ing ens ures
that out put fro m each goroutine ispro cessedasaunit, wit h no dangerofint erleaving iftwo
goro utines finish atthe sametime.
Exercis e 1.10: Findaweb sit e that pro duces a large amountofdat a. Invest igatecachingby
running fetchall twice in successiontosee whether the rep orted time changesmuch. Do
youget the samecontent eachtime? Modif y fetchall to print its out put toafile soitcan be
examined.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 1.7. AWEB SERVER 19
Exercis e 1.11: Tr y fetchall with lon g erargumentlists, suchassamples fro m thetop million
we b sitesavai lable at alexa.com.How doesthe program beh ave ifawe b site justdoesnt
resp ond?(Section8.9 descr ibesmechanismsfor coping in suchcas es.)
1.7. A WebServer
Goslibrar ies makesiteasy towrite a web ser ver thatrespond s to clientrequests like those
made by fetch.Inthissec tion,we ll showaminimal ser ver thatretur nsthe pat h comp onent
of the URL usedtoaccessthe ser ver.Thatis, if the requestisfor http://local-
host:8000/hello,the respons e wi l l be URL.Path = "/hello".
gopl.io/ch1/server1
// Server1 is a minimal "echo" server.
package main
import (
"fmt"
"log"
"net/http"
)
func main() {
http.HandleFunc("/", handler) // each request calls handler
log.Fatal(http.ListenAndServe("localhost:8000", nil))
}
// handler echoes the Path component of the request URL r.
func handler(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "URL.Path = %q\n", r.URL.Path)
}
Theprogram isonlyahandf uloflines long because librar y func tions domostofthe wor k.
The main func tionconne cts a handler functiontoincomingURLs thatbeg in with /,whichis
al l URLs, andstartsaser ver listeningfor incomingrequests onpor t 8000. A re questisrep-
resent edasastr uct oftyp e http.Request,whichcontainsanumberofrel ate d elds,one of
whichisthe URL ofthe incomingrequest. Whenarequestarrives, itisgiven tothe handler
func tion, whichext racts the pat h comp onent (/hello)fro m therequestURL andsends it
back asthe respons e,using fmt.Fprintf.Web ser verswill beexplained indet ailin
Section7.7.
Letsstart the ser ver in the backg round.OnMac OS X orLinux, add an ampers and (&)tothe
command; onMicros oft Windows, you will need torun the command wit houtthe amp ers and
in a sep aratecommand window.
$gorun src/gopl.io/ch1/server1/main.go &
We can thenmakeclientrequests fro m thecommand line:
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
20 CHAPTER 1. TUTORIAL
$gobuild gopl.io/ch1/fetch
$./fetch http://localhost:8000
URL.Path = "/"
$./fetch http://localhost:8000/help
URL.Path = "/help"
Alternat ive ly, wecan accessthe ser ver fro m aweb brows er, as shown inFigure1.2.
Figure 1.2. Arespons e from the ech o server.
Itseasy toadd featurestothe ser ver.One usefuladdition isaspecificURL thatretur nsa
st atu s of som e sort.For example, thisversiondoesthe sameech o butals o counts the number
of requests; a requesttothe URL /count returnsthe count sofar,exc luding /count re quests
themselves:
gopl.io/ch1/server2
// Server2 is a minimal "echo" and counter server.
package main
import (
"fmt"
"log"
"net/http"
"sync"
)
var mu sync.Mutex
var count int
func main() {
http.HandleFunc("/", handler)
http.HandleFunc("/count", counter)
log.Fatal(http.ListenAndServe("localhost:8000", nil))
}
// handler echoes the Path component of the requested URL.
func handler(w http.ResponseWriter, r *http.Request) {
mu.Lock()
count++
mu.Unlock()
fmt.Fprintf(w, "URL.Path = %q\n", r.URL.Path)
}
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 1.7. AWEB SERVER 21
// counter echoes the number of calls so far.
func counter(w http.ResponseWriter, r *http.Request) {
mu.Lock()
fmt.Fprintf(w, "Count %d\n", count)
mu.Unlock()
}
Theser ver has two handlers, andthe requestURL deter mines whichone iscal le d: arequest
for /count invo kes counter andall othersinv oke handler.Ahand ler pattern that ends wit h
aslash match esany URL thathas the pattern asaprefix. Behindthe scenes, the ser ver runs
thehandler for eachincomingrequestinasep arategoroutine sothatitcan ser vemultiple
re quests simultane ously.How ever, iftwo con cur rentrequests try toupdate count at the same
time,itmig htnot beincrement edcon sistent ly; the program wou ldhaveaser ious bug cal le d a
ra c e condit ion (§9.1). Toavoid thispro blem, wemustens ure that at mostone goroutine
accessesthe var iable at a time, whichisthe pur pos e of the mu.Lock() and mu.Unlock() calls
that bracketeachaccessof count.Well lookmoreclos ely atcon cur rency wit h sh are d var i-
ables in Chapt er9.
As a richerexample, the handler functioncan rep ort onthe headersand for m data that it
re ceives, mak ingthe ser ver usefulfor ins pec ting and debug gingrequests:
gopl.io/ch1/server3
// handler echoes the HTTP request.
func handler(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "%s %s %s\n", r.Method, r.URL, r.Proto)
for k, v := range r.Header {
fmt.Fprintf(w, "Header[%q] = %q\n", k, v)
}
fmt.Fprintf(w, "Host = %q\n", r.Host)
fmt.Fprintf(w, "RemoteAddr = %q\n", r.RemoteAddr)
if err := r.ParseForm(); err != nil {
log.Print(err)
}
for k, v := range r.Form {
fmt.Fprintf(w, "Form[%q] = %q\n", k, v)
}
}
Thisusesthe elds ofthe http.Request st ruc t to pro duce out put likethis:
GET /?q=query HTTP/1.1
Header["Accept-Encoding"] = ["gzip, deflate, sdch"]
Header["Accept-Language"] = ["en-US,en;q=0.8"]
Header["Connection"] = ["keep-alive"]
Header["Accept"] = ["text/html,application/xhtml+xml,application/xml;..."]
Header["User-Agent"] = ["Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_5)..."]
Host = "localhost:8000"
RemoteAddr = "127.0.0.1:59911"
Form["q"] = ["query"]
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
22 CHAPTER 1. TUTORIAL
No tice how the cal l to ParseForm is neste d within an if st atement.Goallowsasimplestate-
ment suchasalocal variabledeclarat iontopre cedethe if condit ion,whichispar tic ularly
us efulfor error handling as in thisexample. Wecou ldhavewritt enitas
err := r.ParseForm()
if err != nil {
log.Print(err)
}
butcom biningthe statementsisshorter andreduces the scope ofthe var iable err,whichis
go o d prac tice.Well define scope inSec tion 2.7.
In these programs, weve seenthree ver y dif ferenttyp es us edasout put streams.The fetch
prog ram copied HTTPrespons e data to os.Stdout,afile,asdid the lissajous prog ram.
The fetchall prog ram thre w therespons e away (whi le counting its lengt h) by copying itto
thetrivialsin k ioutil.Discard.And the web ser ver abov e us ed fmt.Fprintf to write to an
http.ResponseWriter repres enting the web brows er.
Although these three typ es dif fer in the det ails ofwhatthe y do,the y al l satisf y acommon
interface,allow ing any ofthemtobeusedwhere ver an out put streamisneeded.Thatint er-
face,cal le d io.Writer,isdis cussed in Sec tion 7.1.
Gosint erface mechanism isthe topic ofChapt er7,but togiveanide a of whatitscap able of,
letssee how easy itistocom bine the web ser ver wit h the lissajous func tionsothatani-
mate d GIFs arewritt ennot tothe stand ard out put,but tothe HTTPclient. Justadd these
lines tothe web ser ver :
handler := func(w http.ResponseWriter, r *http.Request) {
lissajous(w)
}
http.HandleFunc("/", handler)
or equivalently:
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
lissajous(w)
})
Thesecon d argumenttothe HandleFunc func tioncal l immediate lyabove isafunc tionlit era l,
that is, an anonymou s func tiondefine d at its point ofuse.Wewill explain itfur therin
Section5.6.
Once youve made thischange , visit http://localhost:8000 in yourbrows er.Eachtimeyou
lo ad thepage, youll see a new animat ionlikethe one inFigure1.3.
Exercis e 1.12: Mo dif y theLissajous ser ver toreadparameter values fro m theURL. For exam-
ple, you mig htarrange itsothataURL like http://localhost:8000/?cycles=20 sets the
numb erofcyc les to 20 ins teadofthe defau lt5.Use the strconv.Atoi func tiontoconvert the
st ringparameter into anint eger. You can see its document ation wit h go doc strconv.Atoi.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 1.8. LOOSE ENDS 23
Figure 1.3. Animated Lissajous figures in a brows er.
1.8. Loose Ends
Thereisalot moretoGothanweve cov ere d in thisquickint roduc tion. Hereare som e topics
weve barelytouch eduponoromitt edent ire ly, wit h ju stenoug h discussionthatthe y wi l l be
fami liar whenthe y make brief appearances beforethe full treatment.
Contro l ow : We cov ere d thetwo fundament alcontrol-flow statements, if and for,but not
the switch st atement,whichisamulti-way branch.Heresasmall example:
switch coinflip() {
case "heads":
heads++
case "tails":
tails++
default:
fmt.Println("landed on edge!")
}
Theresultofcal ling coinflip is compare d to the value ofeachcas e.Cas es areevaluate d from
toptobot tom,sothe rs t matchingone isexe cut ed. The opt ion aldefau ltcas e match esifnon e
of the other cas es do es; it may beplace d anywhere . Casesdonot fal l thro ugh fro m on e to the
next as in C-like langu ages (thoug h thereisararelyused fallthrough st atement thatover-
rides thisbeh avior).
A switch do es notneed anoperand; itcan justlistthe cas es, each ofwhichisaboole an
expression:
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
24 CHAPTER 1. TUTORIAL
func Signum(x int) int {
switch {
case x > 0:
return +1
default:
return 0
case x < 0:
return -1
}
}
Thisfor m is cal le d a taglessswitch;itsequivalentto switch true.
Like the for and if st atements, a switch mayinclude an opt ion alsimplestatementashort
var iable decl arat ion, an incrementorassig nmentstatement,orafunctioncal l—that can be
us edtoset a value beforeitisteste d.
The break and continue st atementsmodif y theflow ofcontrol . A break caus escontrol to
resume atthe next statement after the inner most for, switch,or select st atement (which
well see later), andaswesaw in Sec tion 1.3, a continue caus esthe inner most for lo opto
st art its next iterat ion. Statementsmay belab ele d so that break and continue canrefer to
them, for ins tance tobre akout of several neste d lo ops at onceortostart the next iterat ionof
theout ermostloop. There isevenagoto st atement,thoug h itsint ended for machine-gener-
ated code, not regu lar use byprogrammers.
Na med types: A type de clarat ionmakes itpossibletogiveaname toanexist ing typ e.Since
st ruc t typesare often long , they are nearlyalways named.Afami liar exampleisthe definition
of a Point type for a 2-D graphics system:
type Point struct {
X, Y int
}
var p Point
Type declarat ions and named typ es arecov ere d in Chapt er2.
Po int ers: Go provides point ers,thatis, values thatcontain the addressofavar iable.Insom e
languages, not ably C,point ers are relative lyuncon strained.Inother langu ages, point ers are
disguisedas ‘‘references,’’ andtheresnot muchthatcan bedon e with them exceptpassthem
around.Gotakes a position som ewhereinthe midd le.Point ers are explicitlyvisible.The &
op erator yieldsthe addressofavar iable,and the * op erator ret rie ves the var iable thatthe
pointerrefersto, but there isnopoint erarithmetic. Well explain point ers inSec tion 2.3.2.
Me thods and inte rfa ces: Amet hod isafunctionass oci ated wit h anamed typ e; Go isunu sual
in thatmet hodsmay beatt ach edtoalmostany named typ e.Met hodsare cov ere d in Chap-
ter6.Int erfaces areabstrac t typesthatlet ustre atdif ferentcon crete typ es in the sameway
basedonwhatmet hodsthe y have , nothow the y arerepresent edorimp lemente d. Interfaces
arethe subjec t of Chapt er7.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 1.8. LOOSE ENDS 25
Pa ckage s: Go com eswit h an extensive stand ard librar y of usefulpackages, andthe Gocom-
munity has create d andshare d many more. Programmingisoften moreabout usingexist ing
packages thanabout writing originalcodeofonesown.Throughoutthe book, wewill point
outacoupleofdozen ofthe most imp ortantstand ard packages, but there are manymorewe
donthavespace tomention,and wecannot provide anyt hingremot ely likeacompleterefer-
ence for any package.
Before you emb ark onany new program, itsagood ide a to see ifpackages already exist that
mig hthelpyou get yourjob don e more easi ly. You can nd anindex ofthe stand ard librar y
packages at https://golang.org/pkg andthe packages cont ribut edbythe community at
https://godoc.org.The go doc to olmakes these documents easi lyaccessiblefro m the
commandline:
$godoc http.ListenAndServe
package http // import "net/http"
func ListenAndServe(addr string, handler Handler) error
ListenAndServe listens on the TCP network address addr and then
calls Serve with handler to handle requests on incoming connections.
...
Comm ents: We havealready mention eddocumentation commentsatthe beg inningofa
prog ram or package. Itsals o go o d st yle towrite a comment beforethe declarat ionofeach
func tiontospecif y itsbeh avior.These convent ion s areimp ortant, because the y areusedby
to ols like go doc and godoc to locateand displ aydocumentation (§10.7.4).
Fo r comments thatspanmultiplelines orapp ear wit hin an expressionorstatement,there is
also the /* ... */ notation fami liar fro m ot her langu ages. Suchcommentsare som etimes
us edatthe beg inningofafile for a large blo ckofexplanatory text toavoid a // on every line.
Wi thin a comment, // and /* have nospeci al me aning, socomments do not nest.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
This page intentionally left blank
From the Library of YIGUANG HU
ptg16105617
2
ProgramStructure
In Go, asinany other programminglangu age, one bui ldslarge programsfro m asmall set of
basic cons tructs. Var iables store values. Simpleexpressions are com bine d into largerones
with operat ions likeaddition and subtrac tion. Basic typ es arecol lec ted int o ag gregates like
ar rays andstr ucts. Expressions are usedinstatementswhose exe cut ion order isdeter mined
by control-flow statementslike if and for.Statementsare gro upedint o func tions for
is olation and reuse.Functions are gat hered int o source files andpackages.
We saw examples ofmostofthese inthe pre vious chapt er. Inthischapt er, well goint o more
det ailabout the basic str uctural elements ofaGoprogram. Theexampleprogramsare int en-
tion allysimple, sowecan focus onthe langu agewit houtgetting sidet racke d by complic ated
algor it hms ordat a st ruc tures.
2.1. Names
Thenames ofGofunctions,var iables, cons tants, typ es, st atement lab els,and packages fol low a
simplerule: a namebeg inswit h aletter(that is, anyt hingthatUnico de de emsa letter) oran
underscoreand may haveany numberofaddition alletters,dig its, andunderscores. Cas e mat-
ters: heapSort and Heapsort aredif ferentnames.
Go has 25 ke yword s li ke if and switch that may beusedonlywhere the syntaxper mits; the y
cantbeused as names.
break default func interface select
case defer go map struct
chan else goto package switch
const fallthrough if range type
continue for import return var
27
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
28 CHAPTER 2. PROGRAM STRUCTURE
In addition,there are about three dozen pred eclare d nameslike int and true forbui lt-in con-
st ants, typ es, andfunctions:
Cons tants: true false iota nil
Types: int int8 int16 int32 int64
uint uint8 uint16 uint32 uint64 uintptr
float32 float64 complex128 complex64
bool byte rune string error
Func tions: make len cap new append copy close delete
complex real imag
panic recover
Thes e namesare not res erve d,soyou may use themindeclarat ions.Well see a handf ulof
pl aces where rede claring one ofthemmakes sense,but bewareofthe pot ent ial for confusion.
If anent ity isdeclare d within a function, itis local to thatfunction. Ifdeclare d outside ofa
func tion, how ever, itisvisible in all files ofthe packagetowhichitbelon gs. Thecas e of the
rs t letterofanamedeter mines its visibilit y acrosspackagebound aries. Ifthe namebeg ins
with anupp er-case letter, itis expor ted,whichmeans thatitisvisible andaccessibleoutside of
itsown packageand may berefer red tobyother par ts of the program, as wit h Printf in the
fmt package. Packagenames thems elves arealways in lowercas e.
Thereisnolimitonnamelengt h, butconvent ion and sty leinGoprogramsleantowardshort
names, especi ally for local variables wit h smal l scop es; you are muchmorelikelytosee var i-
ables named i than theLoopIndex.General ly, the largerthe scope ofaname, the lon g erand
more meaningf ul it shouldbe.
St ylist ically, Goprogrammersuse ‘‘camel cas e’’ when for mingnames bycom biningwords;that
is,int erior capit alletters are preferred overint erior underscores. Thus the stand ard librar ies
have functions wit h nameslike QuoteRuneToASCII and parseRequestLine butnever
quote_rune_to_ASCII or parse_request_line.The letters ofacronymsand initialismslike
ASCII andHTMLare always rendered inthe samecas e,soafunctionmig htbecal le d html-
Escape, HTMLEscape,or escapeHTML,but not escapeHtml.
2.2. Declarations
A decl arati on namesaprogram entity and specifies som e or all ofits pro per ties. Thereare
four maj orkinds ofdeclarat ions: var, const, type,and func.Well tal k ab out var iables and
types in thischapt er, con stants in Chapt er 3, and functions inChapt er5.
AGoprogram isstore d in one ormorefiles whose names endin .go.Each file beg inswit h a
package de clarat ionthatsays whatpackagethe file ispar t of . The package de clarat ionis
fo llowe d by any import de clarat ions,and thenasequence of pack age-level de clarat ions of
types, variables, cons tants, andfunctions,inany order.For example, thisprogram decl aresa
cons tant, a function, andacoupleofvar iables:
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 2.2. DECLARATIONS 29
gopl.io/ch2/boiling
// Boiling prints the boiling point of water.
package main
import "fmt"
const boilingF = 212.0
func main() {
var f = boilingF
var c = (f - 32) * 5 / 9
fmt.Printf("boiling point = %g°F or %g°C\n", f, c)
// Output:
// boiling point = 212°F or 100°C
}
Thecon stant boilingF is a pac
kage-le vel declarat ion(as is main), where asthe var iables f and
c arelocal tothe function main.The nameofeachpackage-le vel ent ity isvisible not only
thro ughoutthe sourcefile thatcontainsits decl arat ion, but throughoutall the files ofthe pack-
age. Bycontrast, localdeclarat ions are visible onlywit hin the functioninwhichthe y are
de clare d andperhaps onlywit hin a smal l part ofit.
Afunctiondeclarat ionhas a name, a listofparameters(thevar iables whose values are
prov ide d by the functionscal lers), an opt ion allistofresults, andthe functionbody, which
cont ainsthe statementsthatdefine whatthe functiondoes. Theresultlistisomitt edifthe
func tiondoesnot retur n anyt hing. Exe cut ion of the functionbeg inswit h thefirs t st atement
andcontinues until itencount ers a retur n st atement orreach esthe end ofafunctionthathas
no results. Control and any results arethenretur ned tothe cal ler.
Weve seenafair numberoffunctions already and there are lotsmoretocom e,includingan
extensive dis cussioninChapt er5,sothisisonlyasketch.The function fToC belowenc apsu-
latesthe temperatureconversionlog ic so thatitisdefine d on lyoncebut may beusedfro m
mu ltipleplaces. Here main callsittwice,usingthe values oftwo dif ferentlocal cons tants:
gopl.io/ch2/ftoc
// Ftoc prints two Fahrenheit-to-Celsius conversions.
package main
import "fmt"
func main() {
const freezingF, boilingF = 32.0, 212.0
fmt.Printf("%g°F = %g°C\n", freezingF, fToC(freezingF)) // "32°F = 0°C"
fmt.Printf("%g°F = %g°C\n", boilingF, fToC(boilingF)) // "212°F = 100°C"
}
func fToC(f float64) float64 {
return (f - 32) * 5 / 9
}
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
30 CHAPTER 2. PROGRAM STRUCTURE
2.3. Variables
A var de clarat ioncre atesavar iable ofapar tic ulartyp e,att ach esanametoit, andsets its ini-
tial value.Eachdeclarat ionhas the general for m
var
name type
=
expression
Either the typ e or the =
expression
part may beomitt ed, but not bot h.Ifthe typ e is omitt ed,
it isdeter mined bythe initializer expression. Ifthe expressionisomitt ed, the initial value is
the zero value forthe typ e,whichis 0 fornumbers, false forboole ans, "" forstr ings, and nil
forint erfaces andreference typ es (slice,point er, map,channel,function). Thezerovalue ofan
ag gregatetyp e li ke an array orastr uct has the zerovalue ofall ofits elements or elds.
Thezero-value mechanism ensuresthatavar iable always holds a wel l-define d value ofits typ e;
in Gothere isnosuchthingasanuninitialize d var iable.Thissimplifies code andoften
ensuressensiblebeh avior of bound ary con dit ion s withoutext ra work.For example,
var s string
fmt.Println(s) // ""
pr intsanemp tystr ing , rat her thancausingsom e kind oferror or unp redic table beh avior.Go
prog rammersoften gotosom e ef for t to makethe zerovalue ofamorecomplic ated typ e
me aningf ul,sothatvar iables beg in life in a usefulstate.
It ispossibletodeclare and opt ion allyinitialize a set ofvar iables in a singledeclarat ion, wit h a
matchinglistofexpressions.Omitt ing the typ e al lows decl arat ionofmultiplevar iables ofdif-
ferent typ es:
var i, j, k int // int, int, int
var b, f, s = true, 2.3, "four" // bool, float64, string
Init ializersmay belit eral values orarbit rar y expressions.Package-le vel var iables areinitial-
ize d before main begins (§2.6.2), andlocal variables areinitialize d as their decl arat ions are
encountere d during functionexe cut ion.
Aset ofvar iables can also beinitialize d by cal lingafunctionthatretur nsmultiplevalues:
var f, err = os.Open(name) // os.Open returns a file and an error
2.3.1. Short Variable Declarations
Wi thin a function, an alternatefor m called a sh ort var iab ledeclarati on maybeusedtodeclare
andinitialize localvar iables. Ittakes the for m
name
:=
expression
,and the typ e of
name
is
deter mined bythe typ e of
expression
.Hereare three ofthe manyshort var iable decl arat ions
in the lissajous func tion(§1.4):
anim := gif.GIF{LoopCount: nframes}
freq := rand.Float64() * 3.0
t:=0.0
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 2.3. VARIABLES 31
Becaus e of their bre vit y and flexibi lit y,short var iable decl arat ions are usedtodeclare and ini-
tialize the maj ority oflocal variables. A var de clarat iontends toberes erve d forlocal variables
that need anexplicittyp e that differs fro m that ofthe initializer expression, orfor whenthe
var iable will beassig ned a value later andits initial value isunimportant.
i:=100 // an int
var boiling float64 = 100 // a float64
var names []string
var err error
var p Point
As wit h var de clarat ions,multiplevar iables may bedeclare d andinitialize d in the sameshort
var iable decl arat ion,
i, j := 0, 1
butdeclarat ions wit h mu ltipleinitializer expressions shouldbeusedonlywhenthe y helpread-
abilit y,such as for short and natural gro upingslikethe initializat ionpar t of a for lo op.
Ke epinmindthat := is a declarat ion, where as = is anassig nment. A mu lti-var iable decl ara-
tion shouldnot beconfusedwit h a tupl e assig nment (§2.4.1), in whicheachvar iable onthe
lef t-handside isassig ned the cor respondingvalue fro m therig ht-handside:
i, j = j, i // swap values of i and j
Like ordinar y var de clarat ions,short var iable decl arat ions may beusedfor cal lstofunctions
li ke os.Open that retur n twoormorevalues:
f, err := os.Open(name)
if err != nil {
return err
}
// ...use f...
f.Close()
Onesubtlebut imp ortantpoint:ashort var iable decl arat iondoesnot necessarily decl are al l the
var iables onits left-handside.Ifsom e of themwerealready declare d in the same lexic al block
(§2.7), thenthe short var iable decl arat ionactslikean assig nment to those var iables.
In the codebelow,the rs t st atement declaresbot h in and err.The secon d de clares out but
on lyassig nsavalue tothe exist ing err var iable.
in, err := os.Open(infile)
// ...
out, err := os.Create(outfile)
Ashort var iable decl arat ionmustdeclare atleast one newvar iable,how ever, sothiscodewill
notcompi le:
f, err := os.Open(infile)
// ...
f, err := os.Create(outfile) // compile error: no new variables
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
32 CHAPTER 2. PROGRAM STRUCTURE
The xistouse anordinar y assig nmentfor the secon d st atement.
Ashort var iable decl arat ionactslikeanassig nmentonlytovar iables thatwerealready
de clare d in the samelexic al block; decl arat ions inanout erblo ckare ignored.Well see exam-
ples ofthis at the end ofthe chapt er.
2.3.2. Pointers
A variab le is a pie ce of storagecontainingavalue.Var iables create d by declarat ions are iden-
tied byaname, suchas x,but manyvar iables areidentied onlybyexpressions like x[i] or
x.f.All these expressions readthe value ofavar iable,exceptwhenthe y appear onthe lef t-
hand side of an assig nment, in whichcas e anew value isassig ned tothe var iable.
A pointer value isthe ad dre ss of a var iable.Apointeristhu s thelocat ionatwhichavalue is
stored.Not every value has an address, but every var iable does. Wit h apoint er, wecan read
or update the value ofavar iable in direc tly,wit houtusingorevenknowing the nameofthe
var iable, if indeed ithas a name.
If a var iable isdeclare d var x int,the expression &x (‘‘addressof x’’)yieldsapoint ertoan
integervar iable,thatis, a value oftyp e *int,whichispro nounced ‘‘pointertoint.’’ If this
value iscal le d p,wesay ‘‘p points to x,’’ or equivalently ‘‘p cont ainsthe addressof x.’’ Thevar i-
able towhich p points iswritt en *p.The expression *p yields the value ofthatvar iable,an
int,but since *p denot esavar iable,itmay also app ear onthe lef t-handside ofanassig nment,
in whichcas e theassig nmentupdates the var iable.
x:=1
p:=&x//p,oftype *int, points to x
fmt.Println(*p) // "1"
*p = 2 // equivalent to x = 2
fmt.Println(x) // "2"
Each component ofavar iable ofaggregatetyp e—a eldofastr uct oranelementofanarray—
is als o avar iable andthu s has an addresstoo.
Variables aresom etimesdes crib edas ad dre ssabl e values. Expressions thatdenot e var iables are
theonlyexpressions towhichthe ad dre ss-of op erator & maybeapp lie d.
Thezerovalue for a point erofany typ e is nil.The test p!=nil is trueif p points toavar i-
able.Point ers are comparable; two point ers are equ alifand onlyifthe y pointtothe same
var iable orbot h arenil.
var x, y int
fmt.Println(&x == &x, &x == &y, &x == nil) // "true false false"
It isper fec tly safefor a functiontoretur n theaddressofalocal variable. For ins tance,inthe
co de below, the local variable v create d by thispar tic ularcal l to f wi l l remain in existence even
af ter the cal l hasretur ned,and the point er p wi l l st i l l refertoit:
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 2.3. VARIABLES 33
var p = f()
func f() *int {
v:=1
return &v
}
Each cal l of f returnsadistinc t value:
fmt.Println(f() == f()) // "false"
Becaus e apoint ercontainsthe addressofavar iable,passingapoint erargumenttoafunction
makesitpossiblefor the functiontoupdatethe var iable thatwas indirec tly passed. For exam-
ple, thisfunctionincrementsthe var iable thatits argumentpointstoand retur nsthe new
value ofthe var iable soitmay beused in an expression:
func incr(p *int) int {
*p++ // increments what p points to; does not change p
return *p
}
v:=1
incr(&v) // side effect: v is now 2
fmt.Println(incr(&v)) // "3" (and v is 3)
Each timewetakethe addressofavar iable orcopyapoint er, wecre ate new aliases or ways to
identify the samevar iable.For example, *p is anali as for v.Point erali asingisusefulbecause
it allowsustoaccessavar iable wit houtusingits name, but thisisadou ble-e dged sword:to
nd all the statementsthataccessavar iable,wehavetoknowall its alias es. Itsnot justpoint-
ersthatcre ate ali ases; aliasingals o occurs whenwecopyvalues ofother reference typ es li ke
slices, maps, andchannel s,and evenstr ucts, arrays, andint erfaces thatcontain these typ es.
Po int ers are key tothe flag package, whichusesaprogramscommand-linearguments toset
thevalues ofcer tain variables dist ribut edthroughoutthe program. Toillustrate, thisvar iat ion
on the earlier echo commandtakes two opt ion al flags: -n caus es echo to omitthe trai ling
ne wlinethatwou ldnor mal lybeprint ed, and -s sep caus esittosep aratethe out put argu-
mentsbythe contentsofthe str ing sep insteadofthe defau ltsinglespace.Since thisisour
four thversion, the packageiscal le d gopl.io/ch2/echo4.
gopl.io/ch2/echo4
// Echo4 prints its command-line arguments.
package main
import (
"flag"
"fmt"
"strings"
)
var n = flag.Bool("n", false, "omit trailing newline")
var sep = flag.String("s", " ", "separator")
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
34 CHAPTER 2. PROGRAM STRUCTURE
func main() {
flag.Parse()
fmt.Print(strings.Join(flag.Args(), *sep))
if !*n {
fmt.Println()
}
}
Thefunction flag.Bool createsanew flag variableoftyp e bool.Ittakes three arguments: the
name ofthe flag ("n"), the var iablesdefau ltvalue (false), andamessage thatwill beprint ed
if the userprovides an invalid argument, an invalid flag , or -h or -help.Simi larly,
flag.String takesaname, a defau ltvalue,and a mess age , andcre atesastring var iable.The
var iables sep and n arepoint ers to the flag variables, whichmustbeaccessedindirec tly as
*sep and *n.
Wh enthe program isrun, itmustcal l flag.Parse before the flags areused, to updatethe flag
var iables fro m their default values. Thenon-flag arguments areavai lable fro m flag.Args()
as a slice ofstr ings. If flag.Parse encounters anerror,itprintsausage message and cal ls
os.Exit(2) to terminatethe program.
Letsrun som e test cas es on echo:
$gobuild gopl.io/ch2/echo4
$./echo4 a bc def
abcdef
$./echo4 -s / a bc def
a/bc/def
$./echo4 -n a bc def
abcdef$
$./echo4 -help
Usage of ./echo4:
-n omit trailing newline
-s string
separator (default " ")
2.3.3. The new Function
Anot her way tocre ate a var iable istouse the bui lt-in function new.The expression new(T)
createsan unname d variab le of typ e T,initializes ittothe zerovalue of T,and retur nsits
address, whichisavalue oftyp e *T.
p:=new(int) // p, of type *int, points to an unnamed int variable
fmt.Println(*p) // "0"
*p = 2 // sets the unnamed int to 2
fmt.Println(*p) // "2"
Avar iable create d with new is nodif ferentfro m an ordinar y lo cal variablewhose addressis
taken, exceptthattheresnoneed toinv ent (anddeclare)adummyname, and wecan use
new(T) in an expression. Thus new is onlyasyntactic conv enience,not a fundament alnot ion:
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 2.3. VARIABLES 35
thetwo newInt func tions below haveidenticalbeh avior s.
func newInt() *int { func newInt() *int {
return new(int) var dummy int
}return &dummy
}
Each cal l to new returnsadistinc t var iable wit h aunique address:
p:=new(int)
q:=new(int)
fmt.Println(p == q) // "false"
Thereisone exception tothisrule: two var iables whose typ e carries noinfor mat ionand is
thereforeofsize zero, suchas struct{} or [0]int,may,dep endingonthe imp lementation,
have the sameaddress.
The new func tionisrel ative lyrarelyusedbecause the most commonunnamed var iables areof
st ruc t types, for whichthe str uct lit eral synt ax(§4.4.1) ismore flexible.
Since new is a pre declare d func tion, not a key word, itspossibletoredefine the namefor
somethingels e within a function, for example:
func delta(old, new int) int { return new - old }
Of course,wit hin delta,the bui lt-in new func tionisunavai lable.
2.3.4. Lifetime ofVariables
The li fetim e of a var iable isthe int erval oftimedur ingwhichitexistsasthe program exec utes.
Thelifet imeofapackage-le vel var iable isthe ent ire exe cut ion of the program. Bycontrast,
lo cal variables havedynamic lifet imes: a new ins tance iscre ate d each timethe declarat ion
st atement isexe cut ed, and the var iable livesonunt i l it becom es unre ach abl e,atwhichpoint its
storagemay berec ycled.Functionparametersand results arelocal variables too;the y arecre-
ated eachtimetheir enclosingfunctioniscal le d.
Fo r example, inthisexcer ptfro m theLissajous program ofSec tion 1.4,
for t := 0.0; t < cycles*2*math.Pi; t += res {
x:=math.Sin(t)
y:=math.Sin(t*freq + phase)
img.SetColorIndex(size+int(x*size+0.5), size+int(y*size+0.5),
blackIndex)
}
thevar iable t is cre ate d each timethe for lo opbeg ins, andnew var iables x and y arecre ate d
on eachiterat ionofthe loop.
Ho w do es thegarb agecol lec tor knowthatavar iablesstoragecan bereclaimed?The full story
is muchmoredet aile d than weneed here, but the basic ideaisthatevery package-le vel var i-
able,and every local variableofeachcur rentlyactivefunction, can pot ent ial lybethe start or
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
36 CHAPTER 2. PROGRAM STRUCTURE
ro otofapat h to the var iable in question,fol low ing point ers and other kinds ofreferences that
ultimate lyleadtothe var iable.Ifnosuchpat h exists, the var iable has becom e unreach able, so
it can nolon g eraffec t therestofthe computation.
Becaus e thelifet imeofavar iable isdeter mined onlybywhether ornot itisreach able, a local
var iable may out liveasingleiterat ionofthe enclosingloop. Itmay cont inuetoexist evenafter
itsenclosingfunctionhas retur ned.
Acompi ler may cho ose toallocatelocal variables onthe heap oronthe stack but,perhaps sur-
pr isingly, thischoice isnot deter mined bywhether var or new was usedtodeclare the var iable.
var global *int
func f() { func g() {
var x int y:=new(int)
x=1 *y = 1
global = &x }
}
Here , x mu stbeheap-al locate d becaus e it isstill reach ablefro m thevar iable global af ter f
hasretur ned,despit e beingdeclare d as a localvar iable; wesay x es cap esfro m f.Convers ely,
when g returns, the var iable *y becomesunreach ableand can berec ycled.Since *y do es not
es cap e from g,itssafefor the compi ler toallocate *y on the stack,eventhoug h it was allo-
cate d with new.Inany cas e,the notion of escapingisnot som ethingthatyou need towor ry
ab out inorder towrite cor rec t co de,thoug h itsgood tokeepinminddur ingper for mance
opt imizat ion, since eachvar iable thatescap es re quires an extra memor y al location.
Garb agecol lec tion isatremendou s helpinwriting cor rec t prog rams, but it doesnot relie ve
youofthe burden ofthin kingabout memor y.You dontneed toexplicitlyallocateand fre e
memory,but towrite efficientprogramsyou still need tobeawareofthe lifet imeofvar iables.
Fo r example, keeping unnecessary point ers to short-live d objec tswit hin long-live d objec ts,
especi ally globalvar iables, will pre ventthe garb agecol lec tor fro m re claimingthe short-live d
objec ts.
2.4. Assignments
Thevalue heldbyavar iable isupdated byanassig nmentstatement,whichinits simplestfor m
hasavar iable onthe lef t of the = sig n and an expressiononthe rig ht.
x=1 // named variable
*p = true // indirect variable
person.name = "bob" // struct field
count[x] = count[x] * scale // array or slice or map element
Each ofthe arithmeticand bit w ise binar y op erator s hasacor responding assig nment operator
al lowing , forexample, the laststatement toberewritt enas
count[x] *= scale
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 2.4. ASSIGNMENTS37
whichsaves usfro m having torep eat (andre-e valuate)the expressionfor the var iable.
Numericvar iables can also beincrement edand decrement edby ++ and -- st atements:
v:=1
v++ // same as v = v + 1; v becomes 2
v-- // same as v = v - 1; v becomes 1 again
2.4.1. Tuple Assignment
Anot her for m of assig nment, known as tupl e assig nment,allowsseveral variables tobe
assig ned atonce. All ofthe rig ht-handside expressions are evaluate d before any ofthe var i-
ables areupdated,mak ingthisfor m most usefulwhensom e of the var iables appear onbot h
sides ofthe assig nment, as happens,for example, whenswapping the values oftwo var iables:
x, y = y, x
a[i], a[j] = a[j], a[i]
or whencomputing the gre atest commondiv isor(GCD) oftwo int egers:
func gcd(x, y int) int {
for y != 0 {
x, y = y, x%y
}
return x
}
or whencomputing the n-t h Fibonacci numberiterat ive ly:
func fib(n int) int {
x, y := 0, 1
for i := 0; i < n; i++ {
x, y = y, x+y
}
return x
}
Tu ple assig nmentcan also makeasequence oftrivialassig nments morecompact,
i, j, k = 2, 3, 5
though asamatterofsty le, avoid the tup lefor m if the expressions are complex; a sequence of
separatestatementsiseasier toread.
Certain expressions,suchasacal l to a functionwit h mu ltipleresults, pro duce several values.
Wh ensuchacal l is used inanassig nmentstatement,the lef t-handside musthaveasmany
var iables as the functionhas results.
f, err = os.Open("foo.txt") // function call returns two values
Of ten, functions use these addition alresults toindic atesom e kind oferror,eit her byretur ning
an error as in the cal l to os.Open,orabool,usu allycal le d ok.Aswell see inlater chapt ers,
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
38 CHAPTER 2. PROGRAM STRUCTURE
thereare three operator s that som etimesbeh ave thisway too.Ifamap lookup (§4.3), typ e
assertion(§7.10), orchannel receive (§8.4.2) appearsinanassig nmentinwhichtwo results are
exp ected,eachpro duces an addition alboole an resu lt:
v, ok = m[key] // map lookup
v, ok = x.(T) // type assertion
v, ok = <-ch // channel receive
As wit h var iable decl arat ions,wecan assig n unwante d values tothe blank identifier :
_, err = io.Copy(dst, src) // discard byte count
_, ok = x.(T) // check type but discard result
2.4.2. Assignability
As sig nmentstatementsare anexplicitfor m of assig nment, but there are manyplaces in a
prog ram whereanassig nmentocc urs implicit ly:a functioncal l implicitlyassig nsthe argument
values tothe cor respondingparameter variables; a return st atement imp licitlyassig nsthe
return op erands tothe cor respondingresultvar iables; andalit eral expressionfor a composite
type (§4.2) such as thisslice:
medals := []string{"gold", "silver", "bronze"}
implicitlyassig nseachelement, as if ithad beenwritt enlikethis:
medals[0] = "gold"
medals[1] = "silver"
medals[2] = "bronze"
Theelements ofmaps andchannel s,thoug h notordinar y var iables, areals o su bjec t to simi lar
implicitassig nments.
An assig nment, explicitorimp licit, isalways legal if the lef t-handside (the var iable) andthe
right-handside (the value) havethe sametyp e.Moregeneral ly, the assig nmentislegal onlyif
thevalue is assig nab le to the typ e of the var iable.
Therulefor assig nab ility hascas es forvar ious typ es, so well explain the relevantcas e as we
introduce eachnew typ e.For the typ es weve dis cussedsofar,the rules aresimple: the typ es
mu stexac tly match,and nil maybeassig ned toany var iable ofint erface orreference typ e.
Cons tants (§3.6) havemore flexiblerules for assig nabilit y that avoidthe need for mostexplicit
conv ersions.
Wh ether two values may becompare d with == and != is rel ate d to assig nabilit y:inany com-
parison, the rs t op erandmustbeassig nable tothe typ e of the secon d op erand, orvice versa.
As wit h assig nabilit y,well explain the relevantcas es for comp arab ility when wepresent each
ne w type.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 2.5. TYPE DECLARATIONS 39
2.5. Type Declarations
Thetyp e of a var iable orexpressiondefinesthe charac ter ist ics ofthe values itmay takeon,
such astheir size (numberofbitsornumberofelements, perhaps), how the y arerepresent ed
internal ly, the int rinsic operat ions thatcan beper for med onthem, andthe methodsass oci-
ated wit h them.
In any program there are var iables thatshare the samerepresent ation but sig nifyver y dif fer-
entcon cepts. For ins tance,an int couldbeusedtorepresent a loopindex, a timestamp,afile
des crip tor,oramonth;afloat64 couldrepresent a velocit y in metersper secon d or a temper-
atureinone ofseveral scales; andastring couldrepresent a passwordorthe nameofacolor.
A type de clarat iondefinesanew name d ty pe that has the same un derly ing type as an exist ing
type.The named typ e prov ides a way tosep aratedif ferentand perhaps incomp atibleusesof
theunderly ing typ e so thatthe y cantbemixed unintent ion ally.
type
name underlying-type
Type declarat ions mostoften appear at packagelevel,where the named typ e is visible through-
outthe package, and ifthe nameisexp orted (it startswit h an upp er-case letter), itsaccessible
from other packages as wel l.
To i llustratetyp e de clarat ions,letstur n thedif ferenttemperaturescales into dif ferenttyp es:
gopl.io/ch2/tempconv0
// Package tempconv performs Celsius and Fahrenheit temperature computations.
package tempconv
import "fmt"
type Celsius float64
type Fahrenheit float64
const (
AbsoluteZeroC Celsius = -273.15
FreezingC Celsius =0
BoilingC Celsius =100
)
func CToF(c Celsius) Fahrenheit { return Fahrenheit(c*9/5 + 32) }
func FToC(f Fahrenheit) Celsius { return Celsius((f - 32) * 5 / 9) }
Thispackagedefinestwo typ es, Celsius and Fahrenheit,for the two units oftemperature.
Ev enthoug h both havethe sameunderly ing typ e, float64,the y arenot the sametyp e,sothe y
cannot becompare d or com bine d in arit hmeticexpressions.Distinguishingthe typ es makesit
possible toavoid erro rslikeinadver tentlycom biningtemperatures in the two dif ferentscales;
an explicittyp e conv ersion li ke Celsius(t) or Fahrenheit(t) is required toconvert from a
float64. Celsius(t) and Fahrenheit(t) areconversions,not functioncal ls. They dont
ch ange the value orrepresent ation inany way,but the y make the change ofmeaningexplicit.
On the other hand, the functions CToF and FToC conv ert bet weenthe two scales; the y do
return dif ferentvalues.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
40 CHAPTER 2. PROGRAM STRUCTURE
Fo r ever y type T,there isacor respondingconversionoperat ion T(x) that conv ertsthe value x
to typ e T.Aconv ersionfro m on e type toanother isallow ed if bot h have the sameunderly ing
type,orifbot h areunnamed point ertyp es that point tovar iables ofthe sameunderly ing typ e;
thes e conv ersions change the typ e butnot the represent ation of the value.If x is assig nable to
T,aconversionisper mitt edbut isusu allyredundant,
Conv ersions are als o al lowe d betweennumer ic types, andbet weenstr ing and som e slice typ es,
as wewill see inthe next chapt er. These conversions may change the represent ation of the
value.For ins tance,convertingafloating-p ointnumbertoanint egerdis cards any frac tional
part,and convertingastr ing toa[]byte slice allo cates a copy ofthe str ing dat a. In any cas e,a
conv ersionnever fails atrun time.
Theunderly ing typ e of a named typ e deter mines its str uctureand represent ation,and als o the
setofint rinsic operat ions itsup por ts, whichare the sameasifthe underly ing typ e hadbeen
us eddirec tly.Thatmeans thatarithmeticoperator s work the samefor Celsius and Fahren-
heit as the y do for float64, as you mig htexp ect.
fmt.Printf("%g\n", BoilingC-FreezingC) // "100" °C
boilingF := CToF(BoilingC)
fmt.Printf("%g\n", boilingF-CToF(FreezingC)) // "180" °F
fmt.Printf("%g\n", boilingF-FreezingC) // compile error: type mismatch
Comp arisonoperator s li ke == and < canals o be usedtocompare a value ofanamed typ e to
anot her ofthe sametyp e,ortoavalue ofthe underly ing typ e.But two values ofdif ferent
name d typescannot becompare d direc tly :
var c Celsius
var f Fahrenheit
fmt.Println(c == 0) // "true"
fmt.Println(f >= 0) // "true"
fmt.Println(c == f) // compile error: type mismatch
fmt.Println(c == Celsius(f)) // "true"!
No tethe lastcas e carefully. Inspit e of itsname, the typ e conv ersion Celsius(f) do es not
ch ange the value ofits argument, justits typ e.The testistruebecause c and f arebot h zero.
Anamed typ e mayprovide not ation alconvenience if ithelps avoidwriting out complex typ es
ov erand overagain. Theadvantage issmall whenthe underly ing typ e is simplelike float64,
butbig for complic ated typ es, as wewill see whenwedis cussstr ucts.
Name d typesals o make itpossibletodefine newbeh avior s forvalues ofthe typ e.These
behavior s areexpress edasaset offunctions ass oci ated wit h thetyp e,cal le d thetyp es metho ds.
Well look at met hods in det ail in Chapt er6but will give a taste ofthe mech anism here.
Thedeclarat ionbelow,inwhichthe Celsius parameter c appearsbeforethe functionname,
asso ciateswit h the Celsius type a met hod named String that retur ns csnumer ic value
fo llowe d by °C:
func (c Celsius) String() string { return fmt.Sprintf("%g°C", c) }
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 2.6. PACKAGES AND FILES 41
Many typ es de clare a String method ofthisfor m becaus e it control s howvalues ofthe typ e
appear whenprint edasastr ing bythe fmt package, aswewill see inSec tion 7.1.
c:=FToC(212.0)
fmt.Println(c.String()) // "100°C"
fmt.Printf("%v\n", c) // "100°C"; no need to call String explicitly
fmt.Printf("%s\n", c) // "100°C"
fmt.Println(c) // "100°C"
fmt.Printf("%g\n", c) // "100"; does not call String
fmt.Println(float64(c)) // "100"; does not call String
2.6. Packages and Files
Packages in Goser vethe samepur pos es as librar ies or modules in other langu ages, sup por ting
mo dular ity,enc apsulat ion, sep aratecompi lat ion, andreuse.The sourcecodefor a package
resides in one ormore .go files, usu allyinadirec tor y whos e name end s with theimp ort pat h;
forins tance,the files ofthe gopl.io/ch1/helloworld packageare store d in direc tor y
$GOPATH/src/gopl.io/ch1/helloworld.
Each packageser ves as a sep arate name space forits decl arat ions.Wit hin the image package,
forexample, the identifier Decode refers to a dif ferentfunctionthandoesthe sameidentifier
in the unicode/utf16 package. Torefer toafunctionfro m outside its package, wemust
qu ali fy theidentifier tomakeexplicitwhether wemean image.Decode or utf16.Decode.
Packages also let ushide infor mat ionbycontrol lingwhichnames arevisible outside the pack-
age, or expor ted.InGo, a simplerulegov ernswhichidentifiersare exp orted and whichare
not: expor ted identifiersstart wit h an upp er-case letter.
To illustratethe basics, sup pos e that our temperatureconversionsof twarehas becom e popu lar
andwewanttomakeitavai lable tothe Gocommunity asanew package. How dowedothat?
Letscre ate a packagecal le d gopl.io/ch2/tempconv,avar iat iononthe pre vious example.
(Hereweve made an exception toour usu alruleofnumbering examples in sequence,sothat
thepackagepat h canbemorerealist ic.) Thepackageits elf isstore d in two files toshowhow
de clarat ions insep arate files ofapackageare accessed; in reallife, a tinypackagelikethis
wouldneed onlyone file.
We haveput the declarat ions ofthe typ es, their cons tants, andtheir met hodsin tempconv.go:
gopl.io/ch2/tempconv
// Package tempconv performs Celsius and Fahrenheit conversions.
package tempconv
import "fmt"
type Celsius float64
type Fahrenheit float64
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
42 CHAPTER 2. PROGRAM STRUCTURE
const (
AbsoluteZeroC Celsius = -273.15
FreezingC Celsius =0
BoilingC Celsius =100
)
func (c Celsius) String() string {return fmt.Sprintf("%g°C", c) }
func (f Fahrenheit) String() string { return fmt.Sprintf("%g°F", f) }
andthe conversionfunctions in conv.go:
package tempconv
// CToF converts a Celsius temperature to Fahrenheit.
func CToF(c Celsius) Fahrenheit { return Fahrenheit(c*9/5 + 32) }
// FToC converts a Fahrenheit temperature to Celsius.
func FToC(f Fahrenheit) Celsius { return Celsius((f - 32) * 5 / 9) }
Each file startswit h a package de clarat ionthatdefinesthe packagename. Whenthe package
is imp orted,its members are referred toas tempconv.CToF andsoon. Package-le vel names
li kethe typ es andcon stantsdeclare d in one file ofa packageare visible toall the other files of
thepackage, asifthe sourcecodewereall inasingle file.Not e that tempconv.go imports fmt,
but conv.go do es not, because itdoesnot use anythingfro m fmt.
Becaus e thepackage-le vel const namesbeg in with upp er-case letters,the y to o areaccessible
with qualified names like tempconv.AbsoluteZeroC:
fmt.Printf("Brrrr! %v\n", tempconv.AbsoluteZeroC) // "Brrrr! -273.15°C"
To c onvert a Cel siu s temp eraturetoFahren heitinapackagethatimp orts gopl.io/ch2/temp-
conv,wecan write the fol low ing code:
fmt.Println(tempconv.CToF(tempconv.BoilingC)) // "212°F"
The doc comment (§10.7.4) immediate lypre cedingthe package de clarat iondocuments the
packageasawhole.Convent ion ally, itshouldstart wit h asummar y sent enceinthe sty le
illustrated.Onlyone file in eachpackageshouldhaveapackagedoc comment.Extensive doc
comments areoften place d in a file oftheir own,convent ion allycal le d doc.go.
Exercis e 2.1: Addtyp es, cons tants, andfunctions to tempconv forpro cessingtemperatures in
theKelvinscale,where zeroKelvinis 273.15°C andadif ference of1Khas the samemag ni-
tude as 1°C.
2.6.1. Imports
Wi thin a Goprogram, every packageisidentied byaunique str ing cal le d its import pat h.
Thes e arethe str ingsthatapp ear in an import de clarat ionlike "gopl.io/ch2/tempconv".
Thelangu agespecification doesntdefine where these str ingscom e from or whatthe y me an;
itsuptothe tools toint erpretthem. Whenusingthe go to ol(Chapter10), an import pat h
denot esadirec tor y cont ainingone ormoreGosourcefiles thattoget her makeupthe package.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 2.6. PACKAGES AND FILES 43
In addition toits import pat h, each packagehas a pack age nam e,whichisthe short (andnot
ne cessarily unique) namethatapp earsinits package de clarat ion. Byconvent ion,apackages
name match esthe lastseg mentofits import pat h, making iteasy topre dic t that the package
name of gopl.io/ch2/tempconv is tempconv.
To use gopl.io/ch2/tempconv,wemustimp ort it:
gopl.io/ch2/cf
// Cf converts its numeric argument to Celsius and Fahrenheit.
package main
import (
"fmt"
"os"
"strconv"
"gopl.io/ch2/tempconv"
)
func main() {
for _, arg := range os.Args[1:] {
t, err := strconv.ParseFloat(arg, 64)
if err != nil {
fmt.Fprintf(os.Stderr, "cf: %v\n", err)
os.Exit(1)
}
f:=tempconv.Fahrenheit(t)
c:=tempconv.Celsius(t)
fmt.Printf("%s = %s, %s = %s\n",
f, tempconv.FToC(f), c, tempconv.CToF(c))
}
}
Theimp ort declarat ionbinds a short nametothe imp orted packagethatmay beusedtorefer
to its cont entsthroughoutthe file.The import ab ove lets usrefer tonames wit hin
gopl.io/ch2/tempconv by usingaqu alied ident ier li ke tempconv.CToF.Bydefau lt, the
short nameisthe packagenametempconv in thiscas ebut an imp ort declarat ionmay
sp ecif y an alternat ive nametoavoid a conflic t (§10.3).
The cf prog ram conv ertsasinglenumer ic command-lineargumenttoits value in bot h
Celsiu s andFahren heit:
$gobuild gopl.io/ch2/cf
$./cf 32
32°F = 0°C, 32°C = 89.6°F
$./cf 212
212°F = 100°C, 212°C = 413.6°F
$./cf -40
-40°F = -40°C, -40°C = -40°F
It isanerror toimp ort a packageand thennot refer toit. Thische ckhelps eliminatedep en-
dencies thatbecom e unnecessary asthe codeevo l ves, although itcan beanuis ancedur ing
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
44 CHAPTER 2. PROGRAM STRUCTURE
debugging, since commenting out a lineofcodelike log.Print("got here!") mayremov e
thesolereference tothe packagename log,causingthe compi ler toemitanerror.Inthissit-
uation,you need tocomment out or deletethe unnecessary import.
Bett erstill,use the golang.org/x/tools/cmd/goimports to ol, whichaut omat ical lyins erts
andremov espackages fro m theimp ort declarat ionasnecessary ; most editors can beconfig-
ured torun goimports each timeyou saveafile.Likethe gofmt to ol, itals o pretty-printsGo
source files in the canonic al format.
Exercis e 2.2: Wr ite a general-pur pos e unit-conv ersionprogram analogou s to cf that reads
numb ers fro m itscommand-linearguments orfro m thestand ard inp utifthere are noargu-
ments, andconvertseachnumberint o units like temperatureinCel siu s andFahren heit,
lengt h in feetand meters,weig ht in pound s andkilog rams, andthe like.
2.6.2. PackageInitialization
Packageinitializat ionbeg insbyinitializingpackage-le vel var iables in the order in whichthe y
aredeclare d,exceptthatdep endencies areres olved rs t:
var a = b + c // a initialized third, to 3
var b = f() // b initialized second, to 2, by calling f
var c = 1 // c initialized first, to 1
func f() int { return c + 1 }
If the packagehas multiple .go files, the y areinitialize d in the order in whichthe files are
giventothe compi ler ; the go to olsor ts .go files bynamebeforeinv oking the compi ler.
Each var iable decl are d at packagelevel startslifewit h thevalue ofits initializer expression, if
any, but for som e var iables, like tables ofdat a, an initializer expressionmay not bethe simplest
way toset its initial value.Inthatcas e,the init func tionmechanism may besimpler.Any
file may cont ain anynumberoffunctions whose declarat ionisjust
func init() { /* ... */ }
Such init func tions cantbecal le d or reference d,but other wis e they are nor mal functions.
Wi thin each file, init func tions are aut omat ical lyexe cut edwhenthe program starts, in the
order in whichthe y aredeclare d.
Onepackageisinitialize d at a time, inthe order ofimp ortsinthe program, dependencies rs t,
so a package p importing q canbesurethat q is fullyinitialize d before psinitializat ionbeg ins.
Init ializat ionpro ceedsfro m thebot tom up;the main packageisthe lasttobeinitialize d.In
this manner,all packages arefullyinitialize d before the app lic ations main func tionbeg ins.
Thepackagebelow definesafunction PopCount that retur nsthe numberofset bits, thatis,
bitswhose value is1,inauint64 value,whichiscal le d its popu lat ion count.Itusesan init
func tiontopre compute a table ofresults, pc,for eachpossible8-bit value sothatthe PopCount
func tionneednttake64steps but can justretur n thesum ofeig httable lookups. (Thisisdefi-
nite ly not thefastest algor it hmfor count ing bits, but itsconvenientfor illustrat ing init
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 2.7. SCOPE 45
func tions,and for showing how topre compute a table ofvalues, whichisoften a useful
prog rammingtechnique.)
gopl.io/ch2/popcount
package popcount
// pc[i] is the population count of i.
var pc [256]byte
func init() {
for i := range pc {
pc[i] = pc[i/2] + byte(i&1)
}
}
// PopCount returns the population count (number of set bits) of x.
func PopCount(x uint64) int {
return int(pc[byte(x>>(0*8))] +
pc[byte(x>>(1*8))] +
pc[byte(x>>(2*8))] +
pc[byte(x>>(3*8))] +
pc[byte(x>>(4*8))] +
pc[byte(x>>(5*8))] +
pc[byte(x>>(6*8))] +
pc[byte(x>>(7*8))])
}
No tethatthe range lo opin init us esonlythe index; the value isunnecessary and thu s ne e d
notbeinclude d.The loopcou ldals o have beenwritt enas
for i, _ := range pc {
Well see other usesof init func tions inthe next sec tion and inSec tion 10.5.
Exercis e 2.3: Re writ e PopCount to use a loopins teadofasingleexpression. Compare the per-
formance ofthe two versions.(Section11.4 shows how tocompare the per for mance ofdif fer-
entimp lementation s systematically.)
Exercis e 2.4: Wr ite a versionof PopCount that counts bitsbyshif tingits argumentthrough 64
bit position s,testing the rig htmostbit eachtime. Compare its per for mance tothe table-
lo oku p version.
Exercis e 2.5: Theexpression x&(x-1) clears the rig htmostnon-zerobit of x.Write a version
of PopCount that counts bitsbyusingthisfac t, andass ess its per for mance.
2.7. Scope
Adeclarat ionass oci ates a namewit h aprogram entity,suchasafunctionoravar iable.The
scop e of a declarat ionisthe par t of the sourcecodewhere a use ofthe declare d name refersto
that decl arat ion.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
46 CHAPTER 2. PROGRAM STRUCTURE
Dontconfuse scop e with lifet ime. The scope ofadeclarat ionisareg ionofthe program text;
it isacompi le-t imepro per ty. The lifet imeofavar iable isthe range oftimedur ingexe cut ion
when the var iable can berefer red tobyother par ts of the program; itisarun-t imepro per ty.
Asyntactic bl o ck is a sequence ofstatementsenclos edinbraces like those thatsur round the
bodyofafunctionorloop. A name declare d inside a synt actic blo ckisnot visible outside that
block. Theblo ckenclos es itsdeclarat ions and deter mines their scope.Wecan generalize this
notion of blo cks toinclude other gro upingsofdeclarat ions thatare not explicitlysur rounded
by braces in the sourcecode; well cal l them all lexicalblocks.There isalexic al blockfor the
entire sourcecode, cal le d the univ ers e bl o ck;for eachpackage; for each file; for each for, if,
and switch st atement;for eachcas e in a switch or select st atement;and,ofcours e, foreach
explicitsyntactic blo ck.
Adeclarat ionslexic al blockdeter mines its scope,whichmay belarge orsmall.The declara-
tion s of bui lt-in typ es, func tions,and con stantslike int, len,and true areinthe univers e
blockand can berefer red tothroughoutthe ent ire program. Declarat ions outside anyfunc-
tion,thatis, at pack age level,can berefer red tofro m any file in the samepackage. Imp orted
packages, suchas fmt in the tempconv example, are declare d at the le level,sothe y canbe
referred tofro m thesame file,but not fro m anot her file in the samepackagewit houtanother
import.Manydeclarat ions,likethatofthe var iable c in the tempconv.CToF func tion, are
local,sothe y canberefer red toonlyfro m within the samefunctionorperhaps justapar t of it.
Thescope ofacontrol-flow lab el, asusedby break, continue,and goto st atements, isthe
entire enclosingfunction.
Aprogram may cont ain multipledeclarat ions ofthe samenamesolon g as eachdeclarat ionis
in a different lexic al block. For example, you can decl are a local variablewit h thesamename
as a package-le vel var iable.Or, asshown inSec tion 2.3.3, you can decl are a functionparame-
tercal le d new,eventhoug h afunctionofthisnameispre declare d in the univers e block. Dont
ov erdoit, thoug h; thelargerthe scope ofthe redeclarat ion, the more likelyyou are tosur prise
thereader.
Wh enthe compi ler encounters a reference toaname, itlooks for a declarat ion, startingwit h
theinner mostenclosinglexic al blockand wor kinguptothe univers e block. Ifthe compi ler
nd s no declarat ion, itrep ortsan ‘‘unde clare d name’’ er ror.Ifanameisdeclare d in bot h an
outerblo ckand aninner blo ck, the inner decl arat ionwill befound rs t.Inthatcas e,the
inner decl arat ionissaid to sh a dow or hi d e theout erone,mak ingitinaccessible:
func f() {}
var g = "g"
func main() {
f:="f"
fmt.Println(f) // "f"; local var f
shadows
package-level func f
fmt.Println(g) // "g"; package-level var
fmt.Println(h) // compile error: undefined: h
}
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 2.7. SCOPE 47
Wi thin a function, lexical blo cks may beneste d to arbit rar y depth,soone local decl arat ioncan
sh adowanother.Mostblo cks arecre ate d by control-flow con str uctslike if st atementsand
for lo ops. Theprogram below has three dif ferentvar iables cal le d x becaus e each declarat ion
appears in a dif ferentlexic al block. (Thisexampleillustrates scope rules, not good sty le!)
func main() {
x:="hello!"
for i := 0; i < len(x); i++ {
x:=x[i]
if x != '!' {
x:=x+'A' - 'a'
fmt.Printf("%c", x) // "HELLO" (one letter per iteration)
}
}
}
Theexpressions x[i] and x+'A' - 'a' each refer toadeclarat ionof x from anout erblo ck;
well explain thatinamom ent.(No tethatthe latterexpressionis not equivalentto uni-
code.ToUpper.)
As mention edabove , notall lexic al blocks correspond toexplicitbrace-delimite d sequences of
st atements; som e aremerelyimp lie d.The for lo opabove cre atestwo lexic al blocks: the
explicitblo ckfor the loopbody, and animp licitblo ckthataddition allyenclos es thevar iables
de clare d by the initializat ionclaus e,suchas i.The scope ofavar iable decl are d in the imp licit
blockisthe con dit ion,post-statement (i++), andbodyofthe for st atement.
Theexamplebelow als o hasthree var iables named x,eachdeclare d in a different blo ckon e
in the functionbody, one inthe for st atementsblo ck, andone inthe loopbodybutonlytwo
of the blo cks areexplicit:
func main() {
x:="hello"
for _, x := range x {
x:=x+'A' - 'a'
fmt.Printf("%c", x) // "HELLO" (one letter per iteration)
}
}
Like for lo ops, if st atementsand switch st atementsals o create imp licitblo cks in addition to
their bodyblo cks. Thecode in the fol low ing if-else ch ain shows the scope of x and y:
if x := f(); x == 0 {
fmt.Println(x)
}else if y := g(x); x == y {
fmt.Println(x, y)
}else {
fmt.Println(x, y)
}
fmt.Println(x, y) // compile error: x and y are not visible here
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
48 CHAPTER 2. PROGRAM STRUCTURE
Thesecon d if st atement isneste d within the rs t,sovar iables decl are d within the rs t st ate-
mentsinitializer arevisible wit hin the secon d.Simi lar rules apply toeachcas e of a switch
st atement:there isablo ckfor the con dit ion and a blo ckfor eachcas e body.
At the packagelevel,the order in whichdeclarat ions app ear has noeffec t on their scope,soa
de clarat ionmay refer toits elf ortoanother thatfol lowsit, letting usdeclare recursiveor
mu tuallyrec ursivetyp es andfunctions.The compi ler will rep ort anerror ifacon stant orvar i-
able decl arat ionreferstoits elf,how ever.
In thisprogram:
if f, err := os.Open(fname); err != nil { // compile error: unused: f
return err
}
f.ReadByte() // compile error: undefined f
f.Close() // compile error: undefined f
thescope of f is justthe if st atement,so f is not accessibletothe statementsthatfol low,
resu lting incompi ler erro rs. Dep endingonthe compi ler,you may get an addition alerror
reportingthatthe local variable f was never used.
Thus itisoften necessary todeclare f before the con dit ion sothatitisaccessibleafter :
f, err := os.Open(fname)
if err != nil {
return err
}
f.ReadByte()
f.Close()
Yo u maybetempt edtoavoid decl aring f and err in the out erblo ckbymov ing the cal lsto
ReadByte and Close inside an else block:
if f, err := os.Open(fname); err != nil {
return err
}else {
// f and err are visible here too
f.ReadByte()
f.Close()
}
butnor mal prac tice in Goistodealwit h theerror inthe if blockand thenretur n, so thatthe
successf ulexe cut ion pat h is not indente d.
Sh ort var iable decl arat ions demandanawarenessofscope.Con sider the program below,
whichstartsbyobt ainingits cur rentwor kingdirec tor y andsav ingit in a package-le vel var i-
able.Thiscou ldbedon e by cal ling os.Getwd in function main,but it mig htbebettertosep a-
ratethiscon cer n from the primary log ic, especi ally iffai lingtoget the direc tor y is a fat al er ror.
Thefunction log.Fatalf pr intsamessage and cal ls os.Exit(1).
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 2.7. SCOPE 49
var cwd string
func init() {
cwd, err := os.Getwd() // compile error: unused: cwd
if err != nil {
log.Fatalf("os.Getwd failed: %v", err)
}
}
Sinceneither cwd nor err is already declare d in the init func tionsblo ck, the := st atement
de claresbot h of themaslocal variables. Theinner decl arat ionof cwd makesthe out erone
inaccessible, sothe statement doesnot updatethe package-le vel cwd var iable as intended.
CurrentGocompi lersdetec t that the local cwd var iable isnever usedand rep ort thisasan
er ror,but the y arenot str ictly required toper for m this che ck. Fur thermore, a minor change ,
such asthe addition of a log gingstatement thatreferstothe local cwd woulddefeatthe che ck.
var cwd string
func init() {
cwd, err := os.Getwd() // NOTE: wrong!
if err != nil {
log.Fatalf("os.Getwd failed: %v", err)
}
log.Printf("Working directory = %s", cwd)
}
Theglobal cwd var iable remainsuninitialize d,and the app arent lynor mal log out put
obfuscates the bug .
Thereare a numberofways todealwit h this pot ent ial pro blem. Themostdirec t is toavoid :=
by declaring err in a sep arate var de clarat ion:
var cwd string
func init() {
var err error
cwd, err = os.Getwd()
if err != nil {
log.Fatalf("os.Getwd failed: %v", err)
}
}
Weve now seenhow packages, files, decl arat ions,and statementsexpress the str uctureof
prog rams. Inthe next two chapt ers,well look at the str uctureofdat a.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
This page intentionally left blank
From the Library of YIGUANG HU
ptg16105617
3
Basic Data Types
Itsall bits at the bot tom,ofcours e, butcomputers operatefundament allyonfixe d-size num-
bers cal le d word s,whichare int erprete d as integers,floating-p ointnumbers,bit sets, ormem-
or y addresses, thencom bine d into largeraggregates thatrepresent packets, pixel s,por tfo lios,
poetr y,and everythingels e.Gooffersavar ietyofways toorganize dat a, with a spectrumof
data typ es that at one end match the featuresofthe hardwareand atthe other endprovide
what programmersneed toconvenientlyrepresent complic ated dat a st ruc tures.
Gostyp es fal l into fourcategor ies: basi c ty pes, ag gre gat e ty pes, reference types,and interface
ty pes.Basic typ es, thetopic ofthischapt er, include numbers,str ings, andboole ans. Aggregate
typesar rays (§4.1) andstr ucts(§4.4)form morecomplic ated dat a typesbycom biningval-
ues ofseveral simpler ones. Reference typ es areadiverse gro upthatincludes point ers (§2.3.2),
slices (§4.2), maps (§4.3), functions (Chapter5), andchannel s (C hapter8), but whatthe y have
in common isthatthe y al l refertoprogram variables orstate in direc tly,sothatthe effec t of an
op erat ionapp lie d to one reference isobs erve d by all copies ofthatreference.Final ly, well tal k
ab out int erface typ es in Chapt er7.
3.1. Int egers
Gosnumer ic data typ es include several sizes ofint egers,floating-p ointnumbers,and complex
numb ers.Eachnumer ic type deter mines the size andsig nedness ofits values. Letsbeg in with
integers.
Go provides bot h sig ned and unsig ned int egerarithmetic. Thereare fourdistinc t sizes of
sig ned int egers8, 16, 32, and64bitsrepres ente d by the typ es int8, int16, int32,and
int64,and cor respondingunsig ned versions uint8, uint16, uint32,and uint64.
51
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
52 CHAPTER 3. BASIC DATATYPES
Thereare als o twotyp es called just int and uint that arethe natural ormostefficientsize for
sig ned and unsig ned int egers onapar tic ularplatfor m; int is byfar the most widely used
numerictyp e.Bot h thes e typeshavethe samesize,eit her 32 or64bits, but onemustnot make
assump tions about which; different compi lersmay makedif ferentchoices evenonidentical
hard ware.
Thetyp e rune is ansynonym for int32 andconvent ion allyindic ates thatavalue isaUnico de
co de point. Thetwo names may beusedint erc hange ably.Simi larly,the typ e byte is ansyn-
onym for uint8,and emp hasizes thatthe value isapie ce of raw dat a rat her thanasmall
numericquant ity.
Final ly, there isanunsig ned int egertyp e uintptr,whose width isnot specified but issuffi-
cienttoholdall the bitsofapoint ervalue.The uintptr type isusedonlyfor low-le vel
prog ramming, suchasatthe bound ary ofaGoprogram wit h aClibrar y or anoperat ingsys-
tem. Well see examples ofthiswhenwedealwit h the unsafe package in Chapt er13.
Regardlessoftheir size, int, uint,and uintptr aredif ferenttyp es from their explicitlysize d
siblings. Thus int is not the sametyp e as int32,evenifthe natural size ofint egers is32bits,
andanexplicitconversionisrequired touse an int value where an int32 is needed,and vice
vers a.
Signed numbers are repres ente d in 2s-comp lementfor m,inwhichthe hig h-order bit is
reserved for the sig n of the numberand the range ofvalues ofan n-bit numberisfro m 2
n1
to 2
n1
1. Unsig ned int egers use the full range ofbitsfor non-negat ive values andthu s have
therange 0 to2
n
1. For ins tance,the range of int8 is 128 to127, where asthe range of
uint8 is 0 to255.
Gosbinar y op erator s forarithmetic, logic, andcomparisonare liste d here inorder ofdecre as-
ingpre cedence:
*/%<<>>&&^
+-|^
== != < <= > >=
&&
||
Thereare onlyfive level s of precedence for binary operator s.Operator s at the samelevel as-
so ciate to the lef t, so parenthes es mayberequired for clarity,ortomakethe operator s evaluate
in the int ended order in an expressionlike mask & (1 << 28).
Each operator inthe rs t twolines ofthe table abov e,for ins tance +,has a corresponding
assig nment operator li ke += that may beusedtoabbre viate anassig nmentstatement.
Theint egerarithmeticoperator s +, -, *,and / maybeapp lie d to int eger, floating-p oint, and
comp lex numbers,but the remainder operator % applies onlytoint egers.The beh avior of % for
negat ive numbers var ies acrossprogramminglangu ages. InGo, the sig n of the remainder is
always the sameasthe sig n of the div idend, so -5%3 and -5%-3 arebot h -2.The beh avior of /
dep ends onwhether its operands are int egers,so 5.0/4.0 is 1.25,but 5/4 is 1 becaus e integer
divisiontrunc atesthe resulttowardzero.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 3.1. INTEGERS 53
If the resultof an arithmeticoperat ion, whether signe d or unsig ned,has morebitsthancan be
repres ente d in the resulttyp e,itissaid to ov erflow.The hig h-order bitsthatdonot tare
si lentlydis carde d.Ifthe originalnumberisasig ned typ e,the resultcou ldbenegat ive ifthe
lef tmost bit is a 1, as in the int8 examplehere:
var u uint8 = 255
fmt.Println(u, u+1, u*u) // "255 0 1"
var i int8 = 127
fmt.Println(i, i+1, i*i) // "127 -128 1"
Tw o integers ofthe sametyp e maybecompare d usingthe binar y comp arisonoperator s
below; the typ e of a comparisonexpressionisaboole an.
== equalto
!= notequ alto
< lessthan
<= lessthanorequ alto
> greaterthan
>= greaterthanorequ alto
In fac t, al l values ofbasic typ eboole ans, numbers,and str ingsare comp arab le,meaning
that two values ofthe sametyp e maybecompare d usingthe == and != op erator s.Fur ther-
more , integers,floating-p ointnumbers,and str ingsare ordere d by the comparisonoperator s.
Thevalues ofmanyother typ es arenot comparable,and no other typ es areordered.Aswe
encountereachtyp e,well present the rules gov erningthe comp arab ility of itsvalues.
Thereare als o unar y addition and subtrac tionoperator s:
+ unar y positive (no effec t)
- unar y negat ion
Fo r integers, +x is a shorthand for 0+x and -x is a shorthand for 0-x;for floating-p ointand
comp lex numbers, +x is just x and -x is the negat ionof x.
Go als o prov ides the fol low ing bit w ise binar y op erator s,the rs t four ofwhichtre attheir op-
erands asbit patternswit h no con ceptofarithmeticcar ryorsig n:
& bit w ise AND
| bit w ise OR
^ bit w ise XOR
&^ bit cle ar (AND NOT)
<< lef t shif t
>> rightshif t
Theoperator ^ is bit w ise exc lusiveOR(XO R)whenusedasabinar y op erator,but whenused
as a unar y prefixoperator it isbit w ise negat ionorcomplement; thatis, itretur nsavalue wit h
each bit inits operandinv erted.The &^ op erator isbit cle ar (AND NOT): in the expression
z=x&^y,eachbit of z is 0 ifthe cor respondingbit of y is 1;other wis e it equ als the cor-
resp ondingbit of x.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
54 CHAPTER 3. BASIC DATATYPES
Thecodebelow shows how bit w ise operat ions can beusedtoint erpretauint8 value as a
comp act and efficientset of8indep endentbits. Ituses Printfs %b verb toprint a numb ers
binary dig its; 08 mo difies %b (an adverb!) topad the resultwit h zeros toexac tly 8 dig its.
var x uint8 = 1<<1 | 1<<5
var y uint8 = 1<<1 | 1<<2
fmt.Printf("%08b\n", x) // "00100010", the set {1, 5}
fmt.Printf("%08b\n", y) // "00000110", the set {1, 2}
fmt.Printf("%08b\n", x&y) // "00000010", the intersection {1}
fmt.Printf("%08b\n", x|y) // "00100110", the union {1, 2, 5}
fmt.Printf("%08b\n", x^y) // "00100100", the symmetric difference {2, 5}
fmt.Printf("%08b\n", x&^y) // "00100000", the difference {5}
for i := uint(0); i < 8; i++ {
if x&(1<<i) != 0 { // membership test
fmt.Println(i) // "1", "5"
}
}
fmt.Printf("%08b\n", x<<1) // "01000100", the set {2, 6}
fmt.Printf("%08b\n", x>>1) // "00010001", the set {0, 4}
(S ection6.5 shows an implementation of int egersets thatcan bemuchbig ger thanabyte.)
In the shif t op erat ions x<<n and x>>n,the n op eranddeter mines the numberofbit position s
to shif t andmustbeunsig ned;the x op erandmay beunsig ned orsig ned.Arithmetically, a lef t
shif t x<<n is equivalenttomultiplic ationby2
n
andarig htshif t x>>n is equivalenttothe oor
of div isionby2
n
.
Left shif ts ll the vac ated bitswit h zeros, as do rig htshif ts of unsig ned numbers,but rig ht
shif ts of sig ned numbers ll the vac ated bitswit h copies ofthe sig n bit.For thisreason, itis
importanttouse unsig ned arithmeticwhenyoure tre ating anint eger as a bit pattern.
Although Goprovides unsig ned numbers and arithmetic, wetendtouse the sig ned int form
even for quant ities thatcantbenegat ive , such asthe lengt h of anarray,thoug h uint mig ht
seem a moreobv iou s ch oice.Indeed,the bui lt-in len func tionretur nsasig ned int,asinthis
lo opwhichannounces prize medals in reverse order :
medals := []string{"gold", "silver", "bronze"}
for i := len(medals) - 1; i >= 0; i-- {
fmt.Println(medals[i]) // "bronze", "silver", "gold"
}
Thealt ernat ive wouldbecal amitous.If len returned anunsig ned number, then i to o would
be a uint,and the con dit ion i>=0wouldalways betruebydefinition.After the thirditera-
tion,inwhich i==0,the i-- st atement wou ldcause i to becom e not 1, but the maximum
uint value (for example, 2
64
1), andthe evaluation of medals[i] wouldfai l at run time, or
pani c (§5.9), byatt emp tingtoaccess an elementoutside the bound s of the slice.
Fo r this reason, unsig ned numbers tend tobeusedonlywhentheir bit w ise operator s or
peculi ar ar ithmeticoperator s arerequired,aswhenimp lementing bit sets, parsingbinar y file
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 3.1. INTEGERS 55
formats, orfor hashingand crypt ography.The y aretypic ally not usedfor merelynon-negat ive
qu ant ities.
In general,anexplicitconversionisrequired toconvert a value fro m on e type toanother,and
binary operator s forarithmeticand log ic (exceptshif ts) mu sthaveoperands ofthe sametyp e.
Although thisocc asionallyresults in longerexpressions,itals o eliminates a whole class of
problemsand makes programseasier tounderst and.
As anexamplefami liar fro m ot her cont exts, consider thissequence:
var apples int32 = 1
var oranges int16 = 2
var compote int = apples + oranges // compile error
At tempt ing tocompi lethese three declarat ions pro duces an erro r mess age:
invalid operation: apples + oranges (mismatched types int32 and int16)
Thistyp e mismatch can be xe d in several ways, mostdirec tly byconvertingeverythingtoa
common typ e:
var compote = int(apples) + int(oranges)
As des crib edinSec tion 2.5, for every typ e T,the conversionoperat ion T(x) conv ertsthe value
x to typ e T if the conversionisallow ed. Manyint eger-to-integerconversions donot ent ail any
ch ange invalue; the y ju sttel l thecompi ler how toint erpretavalue.But a conversionthatnar-
rows a big integerint o asmaller one,oraconversionfro m integertofloating-p ointorvice
vers a,may change the value orlos e precision:
f:=3.141 // a float64
i:=int(f)
fmt.Println(f, i) // "3.141 3"
f=1.99
fmt.Println(int(f)) // "1"
Flo at to int egerconversiondis cards any frac tionalpar t, tr unc ating towardzero. You should
avoidconversions inwhichthe operandisout of range for the targettyp e,because the beh av-
iordep ends onthe imp lementation:
f:=1e100 // afloat64
i:=int(f) // result is implementation-dependent
Integerlit eralsofany size andtyp e canbewritt enasordinar y de cimal numbers,orasoctal
numb ers ifthe y beginwit h 0,asin 0666,orashexade cimal if the y beginwit h 0x or 0X,asin
0xdeadbeef.Hex digitsmay beupp erorlow ercas e.Nowad ays octalnumbers seemtobe
us edfor exac tly one pur pos efile per missions onPOSIX systemsbuthexade cimal numbers
arewidely usedtoemp hasize the bit pattern ofanumberoverits numer ic value.
Wh enprint ing numbers usingthe fmt package, wecan cont rol the radix andfor mat wit h the
%d, %o,and %x verbs, as shown inthisexample:
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
56 CHAPTER 3. BASIC DATATYPES
o:=0666
fmt.Printf("%d %[1]o %#[1]o\n", o) // "438 666 0666"
x:=int64(0xdeadbeef)
fmt.Printf("%d %[1]x %#[1]x %#[1]X\n", x)
// Output:
// 3735928559 deadbeef 0xdeadbeef 0XDEADBEEF
No tethe use oftwo fmt tr icks. Usu allyaPrintf format str ing containingmultiple % verbs
wouldrequirethe samenumberofext ra op erands,but the [1] ‘‘adverbs’’ af ter % te ll Printf to
us e thefirs t op erandoverand overagain. Secon d,the # adverb for %o or %x or %X te lls Printf
to emita0 or 0x or 0X prefixrespectively.
Rune lit eralsare writt enasacharac ter wit hin singlequotes. ThesimplestexampleisanASCII
ch arac ter like 'a',but itspossibletowrite any Unico de co de pointeit her direc tly orwit h
numericescap es, as wewill see shortly.
Runesare print edwit h %c,orwit h %q if quoting isdesired:
ascii := 'a'
unicode := 'D'
newline := '\n'
fmt.Printf("%d %[1]c %[1]q\n", ascii) // "97 a 'a'"
fmt.Printf("%d %[1]c %[1]q\n", unicode) // "22269 D 'D'"
fmt.Printf("%d %[1]q\n", newline) // "10 '\n'"
3.2. Floating-PointNumbers
Go provides two sizes of floating-p ointnumbers, float32 and float64.Their arit hmetic
prop erties aregov erned bythe IEEE 754 stand ard imp lemente d by all moder n CPUs.
Values ofthese numer ic typesrange fro m tiny tohuge. The limits of floating-p ointvalues can
be found inthe math package. The con stant math.MaxFloat32,the largest float32,isabout
3.4e38,and math.MaxFloat64 is about 1.8e308.The smallestpositive values arenear
1.4e-45 and 4.9e-324,respectively.
A float32 prov ides approximatelysix decimal digitsofpre cision, where asafloat64
prov ides about15dig its; float64 shouldbepreferred for mostpur pos es becaus e float32
comp utation s acc umulateerror rapid lyunlessone isquite caref ul, and the smallestpositive
integerthatcannot beexac tly represent ed as a float32 is not large:
var f float32 = 16777216 // 1 << 24
fmt.Println(f == f+1) // "true"!
Flo at ing-p ointnumbers can bewritt enlit eral lyusingdecimals, like this:
const e = 2.71828 // (approximately)
Dig its may beomitt edbeforethe decimal point (.707)orafter it(1.). Ver y smal l or ver y
large numbers are betterwritt eninscienticnot ation,wit h theletter e or E precedingthe dec-
imal expon ent:
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 3.2. FLOATING-POINT NUMBERS 57
const Avogadro = 6.02214129e23
const Planck =6.62606957e-34
Flo at ing-p ointvalues areconvenientlyprint edwit h Printfs %g verb,whichcho osesthe most
comp act represent ation thathas adequ ate pre cision, but for tables ofdat a, the %e (exp onent)
or %f (noexp onent)for msmay bemoreappro priate. All three verbs allow eldwidth and
numericpre cisiontobecontrol led.
for x := 0; x < 8; x++ {
fmt.Printf("x = %d eA =%8.3f\n", x, math.Exp(float64(x)))
}
Thecodeabove printsthe pow ers of e with thre e de cimal digitsofpre cision, aligne d in an
eig ht-charac ter eld:
x=0 eA =1.000
x=1 eA =2.718
x=2 eA =7.389
x=3 eA =20.086
x=4 eA =54.598
x=5 eA =148.413
x=6 eA =403.429
x=7 eA =1096.633
In addition toalarge col lec tion of the usu almat hemat ical functions,the math packagehas
func tions for cre ating and detec ting the speci al values define d by IEEE 754: the positive and
negat ive infinities, whichrepresent numb ers ofexcessive mag nitude andthe resultofdiv ision
by zero; andNaN (‘‘notanumber’’), the resultofsuchmat hemat ical lydubious operat ions as
0/0 or Sqrt(-1).
var z float64
fmt.Println(z, -z, 1/z, -1/z, z/z) // "0 -0 +Inf -Inf NaN"
Thefunction math.IsNaN testswhether its argumentisanot-a-numbervalue,and math.NaN
returnssuchavalue.Itstempt ing touse NaN as a sentinel value in a numer ic comp utation,
buttesting whether a specificcomputation alresultisequ altoNaN isfraug htwit h peril
becaus e anycomparisonwit h NaN always yields false:
nan := math.NaN()
fmt.Println(nan == nan, nan < nan, nan > nan) // "false false false"
If a functionthatretur nsafloating-p ointresultmig htfai l,itsbettertorep ort the fai luresep a-
rately, likethis:
func compute() (value float64, ok bool) {
// ...
if failed {
return 0, false
}
return result, true
}
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
58 CHAPTER 3. BASIC DATATYPES
Thenext program illustrates floating-p ointgraphics comp utation.Itplots a functionoftwo
var iables z=f(x, y) as a wiremesh3-D sur face,usingScalableVec tor Graphics (SVG), a
st and ard XML not ation for linedraw ings. Figure3.1 shows an exampleofits out put for the
func tion sin(r)/r,where r is sqrt(x*x+y*y).
Figure 3.1. Asur face plot ofthe function sin(r)/r.
gopl.io/ch3/surface
// Surface computes an SVG rendering of a 3-D surface function.
package main
import (
"fmt"
"math"
)
const (
width, height = 600, 320 // canvas size in pixels
cells = 100 // number of grid cells
xyrange = 30.0 // axis ranges (-xyrange..+xyrange)
xyscale = width / 2 / xyrange // pixels per x or y unit
zscale = height * 0.4 // pixels per z unit
angle = math.Pi / 6 // angle of x, y axes (=30°)
)
var sin30, cos30 = math.Sin(angle), math.Cos(angle) // sin(30°), cos(30°)
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 3.2. FLOATING-POINT NUMBERS 59
func main() {
fmt.Printf("<svg xmlns='http://www.w3.org/2000/svg' "+
"style='stroke: grey; fill: white; stroke-width: 0.7' "+
"width='%d' height='%d'>", width, height)
for i := 0; i < cells; i++ {
for j := 0; j < cells; j++ {
ax, ay := corner(i+1, j)
bx, by := corner(i, j)
cx, cy := corner(i, j+1)
dx, dy := corner(i+1, j+1)
fmt.Printf("<polygon points='%g,%g %g,%g %g,%g %g,%g'/>\n",
ax, ay, bx, by, cx, cy, dx, dy)
}
}
fmt.Println("</svg>")
}
func corner(i, j int) (float64, float64) {
// Find point (x,y) at corner of cell (i,j).
x:=xyrange * (float64(i)/cells - 0.5)
y:=xyrange * (float64(j)/cells - 0.5)
// Compute surface height z.
z:=f(x, y)
// Project (x,y,z) isometrically onto 2-D SVG canvas (sx,sy).
sx := width/2 + (x-y)*cos30*xyscale
sy := height/2 + (x+y)*sin30*xyscale - z*zscale
return sx, sy
}
func f(x, y float64) float64 {
r:=math.Hypot(x, y) // distance from (0,0)
return math.Sin(r) / r
}
No tice thatthe function corner returnstwo values, the coordinates ofthe cor ner ofthe cel l.
Theexplanation of how the program wor ksrequires onlybasic geometr y,but itsfine toskip
ov erit, since the point istoillustrate floating-p ointcomputation.The ess enceofthe program
is mappingbet weenthree dif ferentcoordinatesystems,shown inFigure3.2. The rs t is a 2-D
gr idof100&100 cellsidentied byint egercoordinates (i, j), startingat(0, 0) in the far back
corner.Weplotfro m thebacktothe fro ntsothatbackg round polygons may beobs cured by
foreground ones.
Thesecon d co ordinatesystemisameshof3-D floating-p ointcoordinates (x, y, z), where x
and y arelinearfunctions of i and j,transl ate d so thatthe originisinthe center, and scale d by
thecon stant xyrange.The heig ht z is the value ofthe sur face func tion f (x, y).
Thethirdcoordinatesystemisthe 2-D imagecanvas, wit h (0, 0) in the top lef t corner.Points
in thisplane are denot ed(sx, sy). Weuse anisometr icpro jec tion tomap each3-D point
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
60 CHAPTER 3. BASIC DATATYPES
Figure 3.2. Three dif ferentcoordinatesystems.
(x, y, z)ont o the2-D canvas. A pointapp earsfar thertothe rig htonthe canvas the gre aterits
x value orthe sm all er its y value.And a point app earsfar therdow n thecanvas the gre aterits x
value or y value,and the smaller its z value.The ver tic al andhor izont alscale factors for x and
y areder ive d from the sineand cosineofa3angle. The scale factorfor z,0.4, isanarbit rar y
parameter.
Fo r each cel l in the 2-D grid, the main functioncomputesthe coordinates onthe image canvas
of the fourcor nersofthe polygon AB CD,where B correspond s to (i, j)and A, C,and D areits
neig hbors,thenprints an SVG ins tructiontodraw it.
Exercis e 3.1: If the function f returnsanon-finite float64 value,the SVG file will contain
invalid <polygon> elements (although manySVG renderershandlethisgracef ully). Modif y
theprogram toskipinvalid polygons.
Exercis e 3.2: Experiment wit h visu alizat ions ofother functions fro m the math package. Can
youpro duce an eggbox,mogu ls, orasadd le?
Exercis e 3.3: Coloreachpolygonbas edonits heig ht, sothatthe peaks arecolored red
(#ff0000)and the val leys blue(#0000ff).
Exercis e 3.4: Fo llowing the appro ach ofthe Lissajous exampleinSec tion 1.7, cons truct a web
server thatcomputessur faces andwritesSVG dat a to the client. Theser ver mustset the Con-
tent-Type he ader like this:
w.Header().Set("Content-Type", "image/svg+xml")
(Thisstepwas not required inthe Lissajous examplebecause the ser ver usesstand ard
heur ist ics torecog nize common for mats like PNG fro m thefirs t 512 bytes ofthe respons e and
generates the pro per header.) Allow the clienttospecif y values like heig ht, width,and color as
HT TP re questparameters.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 3.3. COMPLEX NUMBERS 61
3.3. ComplexNumbers
Go provides two sizes ofcomplex numbers, complex64 and complex128,whose components
are float32 and float64 resp ectively. The bui lt-in function complex createsacomplex num-
berfro m itsrealand imaginary components, andthe bui lt-in real and imag func tions ext ract
thos e comp onents:
var x complex128 = complex(1, 2) // 1+2i
var y complex128 = complex(3, 4) // 3+4i
fmt.Println(x*y) // "(-5+10i)"
fmt.Println(real(x*y)) // "-5"
fmt.Println(imag(x*y)) // "10"
If a floating-p ointlit eral ordecimal integerlit eral isimmediate lyfol low edby i,suchas
3.141592i or 2i,itbecom esan im aginary lit era l,denot ing a complex numberwit h azeroreal
comp onent:
fmt.Println(1i * 1i) // "(-1+0i)", i$ =-1
Un der the rules for con stant arithmetic, comp lex cons tants can beadde d to other cons tants
(integerorfloating point,realorimaginary), allowing ustowrite complex numbers natural ly,
li ke 1+2i,orequivalently, 2i+1.The declarat ions of x and y ab ove can besimplified:
x:=1+2i
y:=3+4i
Comp lex numbers may becompare d forequ ality with == and !=.Two complex numbers are
equal if their realpar ts areequ aland their imag inar y partsare equ al.
The math/cmplx packageprovides librar y func tions for wor kingwit h comp lex numbers,such
as the complex squ are root and exp onent iat ionfunctions.
fmt.Println(cmplx.Sqrt(-1)) // "(0+1i)"
Thefol low ing program uses complex128 ar ithmetictogenerateaMandelbro t set.
gopl.io/ch3/mandelbrot
// Mandelbrot emits a PNG image of the Mandelbrot fractal.
package main
import (
"image"
"image/color"
"image/png"
"math/cmplx"
"os"
)
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
62 CHAPTER 3. BASIC DATATYPES
func main() {
const (
xmin, ymin, xmax, ymax = -2, -2, +2, +2
width, height =1024, 1024
)
img := image.NewRGBA(image.Rect(0, 0, width, height))
for py := 0; py < height; py++ {
y:=float64(py)/height*(ymax-ymin) + ymin
for px := 0; px < width; px++ {
x:=float64(px)/width*(xmax-xmin) + xmin
z:=complex(x, y)
// Image point (px, py) represents complex value z.
img.Set(px, py, mandelbrot(z))
}
}
png.Encode(os.Stdout, img) // NOTE: ignoring errors
}
func mandelbrot(z complex128) color.Color {
const iterations = 200
const contrast = 15
var v complex128
for n := uint8(0); n < iterations; n++ {
v=v*v + z
if cmplx.Abs(v) > 2 {
return color.Gray{255 - contrast*n}
}
}
return color.Black
}
Thetwo neste d lo ops iterateovereachpoint ina1024&1024 grays cale rasterimage represent-
ingthe 2to+2por tionofthe complex plane.The program tests whether rep eatedlysqu ar-
ingand addingthe numberthatpoint representsevent ual ly ‘‘es cap es’’ thecircleofradius 2.If
so,the point isshade d by the numberofiterat ions ittooktoescap e.Ifnot,the value belon gs
to the Mandelbro t set, andthe point remainsblack.Final ly, the program writestoits stand ard
output the PNG-enco dedimage ofthe iconic frac tal,shown inFigure3.3.
Exercis e 3.5: Implementafull-colorMandelbro t setusingthe function image.NewRGBA and
thetyp e color.RGBA or color.YCbCr.
Exercis e 3.6: Su persamp lingisatechnique toreduce the effec t of pixe lat ionbycomputing the
colorvalue at several pointswit hin eachpixel and tak ingthe average. The simplestmet hod is
to div ide eachpixel int o four ‘‘su bpixe ls.’’ Implementit.
Exercis e 3.7: Anot her simplefrac tal usesNewtonsmet hod tofind complex solut ion s to a
func tionsuchas z
4
1=0.Shade eachstartingpoint bythe numberofiterat ions required to
getclos e to one ofthe fourroots. Color eachpoint bythe rootitappro ach es.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 3.4. BOOLEANS 63
Figure 3.3. TheMandelbro t set.
Exercis e 3.8: Render ingfrac talsathig h zo omlevel s demands gre atarithmeticpre cision.
Implementthe samefrac tal usingfourdif ferentrepresent ation s of numbers: complex64, com-
plex128, big.Float,and big.Rat.(Thelattertwo typ es arefound inthe math/big package.
Float us esarbit rar y butbounded-pre cision floating-p oint; Rat us esunb ounde d-pre cision
rat ionalnumbers.) How dothe y comp are inper for mance andmemor y us age?Atwhatzoom
le vel s do render ingartifac ts become visible?
Exercis e 3.9: Wr ite a web ser ver thatrendersfrac talsand writesthe image dat a to the client.
Al low the clienttospecif y the x, y,and zoomvalues as parameterstothe HTTPrequest.
3.4. Booleans
Avalue oftyp e bool,or boolean,has onlytwo possiblevalues, true and false.The con di-
tion s in if and for st atementsare boole ans, andcomparisonoperator s li ke == and < produce
aboole an resu lt. Theunary operator ! is log ical negat ion, so !true is false,or, one mig ht
say, (!true==false)==true,alt hough asamatterofsty le, wealways simplif y re dundant
boole an expressions like x==true to x.
Booleanvalues can becom bine d with the && (AND) and || (OR) operator s,whichhave sh ort-
circuit behavior : if the answerisalready deter mined bythe value ofthe lef t op erand, the rig ht
op erandisnot evaluate d, making itsafetowrite expressions likethis:
s!=""&&s[0] == 'x'
where s[0] wouldpanic if applie d to anemp tystr ing .
Since && hashig her pre cedence than || (mnemonic: && is boole an mu ltiplic ation, || is
boole an addition), noparenthes es arerequired for con dit ion s of thisfor m:
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
64 CHAPTER 3. BASIC DATATYPES
if 'a' <= c && c <= 'z' ||
'A' <= c && c <= 'Z' ||
'0' <= c && c <= '9' {
// ...ASCII letter or digit...
}
Thereisnoimp licitconversionfro m aboole an value toa numer ic value like 0 or1,orvice
vers a.Itsnecessary touse anexplicit if, as in
i:=0
if b {
i=1
}
It mig htbewor thwriting a conversionfunction if thisoperat ionwereneeded often:
// btoi returns 1 if b is true and 0 if false.
func btoi(b bool) int {
if b {
return 1
}
return 0
}
Theinv ers e op erat ionissosimplethatitdoesntwar rantafunction, but for symmetr y here it
is:
// itob reports whether i is non-zero.
func itob(i int) bool { return i != 0 }
3.5. Strings
Astr ing isanimmut ablesequence ofbytes. Str ingsmay cont ain arbit rar y data,including
byteswit h value 0, but usu allythe y cont ain human-readabletext. Text str ingsare convent ion-
al lyint erprete d as UTF-8-enco dedsequences ofUnico de co de points (runes), whichwell
explore indet ailver y so on.
Thebui lt-in len func tionretur nsthe numberofbytes (not runes) in a str ing , andthe in dex
op erat ion s[i] retr ieves the i-t h byte ofstr ing s,where 0 i < len(s).
s:="hello, world"
fmt.Println(len(s)) // "12"
fmt.Println(s[0], s[7]) // "104 119" ('h' and 'w')
At tempt ing toaccessabyteoutside thisrange results in a panic:
c:=s[len(s)] // panic: index out of range
The i-t h byte ofastr ing isnot necessarily the i-t h ch ara cter of a str ing , becaus e theUTF-8
enco dingofanon-ASCII code point requires two ormorebytes. Wor kingwit h ch arac tersis
discussedshortly.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 3.5. STRINGS 65
The substr ing op erat ion s[i:j] yields a new str ing con sisting ofthe bytes ofthe originalstr ing
st artingatindex i andcontinuingupto, but not including, the byte at index j.The result
cont ains j-i bytes.
fmt.Println(s[0:5]) // "hello"
Again, a panic results if either index isout of bound s or if j is lessthan i.
Either orbot h of the i and j op erands maybeomitt ed, inwhichcas e thedefau ltvalues of 0
(t hestart ofthe str ing)and len(s) (its end) areassume d,respectively.
fmt.Println(s[:5]) // "hello"
fmt.Println(s[7:]) // "world"
fmt.Println(s[:]) // "hello, world"
The + op erator makes a new str ing bycon catenat ingtwo str ings:
fmt.Println("goodbye" + s[5:]) // "goodbye, world"
St rings may becompare d with comparisonoperator s li ke == and <;the comparisonisdon e
byte bybyte, sothe resultisthe natural lexicographic order ing.
St ringvalues areimmut able: the bytesequence cont ained inastr ing value can never be
ch ange d,thoug h of cours e we can assig n anew value toastr ing variab le.Toapp end one
st ringtoanother,for ins tance,wecan write
s:="left foot"
t:=s
s+=",right foot"
Thisdoesnot modif y thestr ing that s or iginallyheldbut causes s to holdthe newstr ing
formed bythe += st atement;meanw hile, t st i l l cont ainsthe old str ing .
fmt.Println(s) // "left foot, right foot"
fmt.Println(t) // "left foot"
Sincestr ingsare immut able, con str uctions thattry tomodif y astr ingsdat a in place arenot
al lowe d:
s[0] = 'L' // compile error: cannot assign to s[0]
Immu tabilit y me ans thatitissafefor two copies ofastr ing toshare the sameunderly ing
memory,mak ingitche aptocopystr ingsofany lengt h. Simi larly,astr ing s andasubst ring
li ke s[7:] maysafelyshare the samedat a, so the subst ringoperat ionisals o ch eap.Nonew
memory isallocated ineit her cas e.Figure3.4 illustrates the arrangement ofastr ing and two
of itssubst rings sharing the sameunderly ing bytearray.
3.5.1. String Literals
Astr ing value can bewritt en as a st ring litera l,asequence ofbytes enclos ed in dou ble quotes:
"Hello, BF"
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
66 CHAPTER 3. BASIC DATATYPES
Figure 3.4. Thestr ing "hello, world" andtwo subst rings.
Becaus e Go source files arealways enco dedinUTF-8 andGotext str ingsare convent ion ally
interprete d as UTF-8, wecan include Unico de co de points in str ing lit erals.
Wi thin a double-quote d st ringlit eral, es cap e sequ ences that beg in with a backslash \ canbe
us edtoins ert arbit rar y byte values into the str ing . Oneset ofescap es hand les ASCII cont rol
co des li kenewline, car r iageretur n, andtab:
\a ‘‘aler t’’ or bel l
\b backsp ace
\f form feed
\n ne wline
\r carriageretur n
\t tab
\v vertic al tab
\' singlequote (on ly in the runelit eral '\'')
\" double quote (on lywit hin "..." literals)
\\ backsl ash
Arbit rar y bytescan also beinclude d in literal str ingsusinghexade cimal oroctal escapes. A
hexadecimalescap e is writt en \xhh,wit h exac tly two hexade cimal digits h (in upp erorlow er
case). An oc tal escap e is writt en \ooo with exac tly three octal digits o (0 through 7)not
exce e ding \377.Bot h denot e asinglebytewit h thespecified value.Later,well see how to
enco de Unico de co de points numer ical ly in str ing lit erals.
A rawstr ing litera l is writt en `...`,usingbackquotesins teadofdou ble quotes. Wit hin a raw
st ringlit eral,noescap e sequences arepro cessed; the contentsare taken literal ly, including
backsl ashesand newlines, soaraw str ing lit eral may spreadoverseveral lines in the program
source.The onlypro cessingisthatcar r iageretur nsare deleted sothatthe value ofthe str ing
is the sameonall platfor ms, includingthose thatconvent ion allyput car r iageretur nsintext
files.
Rawstr ing lit eralsare a convenientway towrite regu lar expressions,whichtendtohavelotsof
backsl ashes. They are als o us efulfor HTMLtempl ates, JSONlit erals, commandusage mes-
sages, andthe like, whichoften extendovermultiplelines.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 3.5. STRINGS 67
const GoUsage = `Go is a tool for managing Go source code.
Usage:
go command [arguments]
...`
3.5.2. Unicode
Long ago,lifewas simpleand there was, at leastinaparochi al view,onlyone charac ter set to
de al with:ASCII, the AmericanStand ard Codefor Infor mat ionInt erc hange . ASCII, ormore
precis ely US-ASCII, uses7bitstorepresent 128 ‘‘ch arac ters’’:the upp er- and low er-case letters
of Eng lish, digits, andavar ietyofpunctuation and device-cont rol charac ters. For muchofthe
earlydays ofcomputing , this was adequ ate, but it lef t aver y large frac tionofthe wor lds
popu lat ionunabletouse their own writ ing systems incomputers.Wit h thegrowt h of the
Internet, dat a in myr iadlangu ages has becom e much morecommon. How can thisrichvar i-
et y be dealt wit h at alland, if possible, efficiently?
TheanswerisUnico de (unicode.org), whichcol lec tsall ofthe charac tersinall ofthe wor lds
wr iting systems,plu s accents andother diacr iticalmarks, cont rol codes like tab andcar r iage
return,and plenty ofesoterica,and assig nseachone a stand ard numbercal le d a Un ico d e co d e
point or, in Go ter minolog y,arune.
Unico de version8definescodepointsfor over120,000 charac tersinwel l ov er100 langu ages
andscr ipts. How are these represent edincomputerprogramsand dat a? Thenatural dat a
type toholdasingleruneis int32,and thatswhatGouses; ithas the synonym rune for
precis ely thispur pos e.
We c ou ldrepresent a sequence ofrunes as a sequence of int32 values. Inthisrepresent ation,
whichiscal le d UTF-32 orUCS-4, the encodingofeachUnico de co de pointhas the samesize,
32 bits. Thisissimpleand unifor m,but it usesmuchmorespace thannecessary since most
comp uter-re adabletext isinASCII, whichrequires only8bitsor1byteper charac ter.All the
ch arac tersinwidespreaduse still numberfewer than65,536, whichwou ldfitin16bits. Can
we dobetter?
3.5.3. UTF-8
UTF-8 isavar iable-lengt h enco dingofUnico de co de points as bytes. UTF-8 was invent edby
KenThompsonand Rob Pike, two ofthe cre ators ofGo, and isnow a Unico de st and ard . It
us esbet ween1and 4 bytes torepresent eachrune, but only 1 bytefor ASCII charac ters, and
on ly2or3bytes for mostrunes in common use.The hig h-order bitsofthe rs t byte ofthe
enco dingfor a runeindic atehow manybytes fol low.Ahig h-order 0 indic ates 7-bit ASCII,
whereeachrunetakes only1byte, soitisidenticaltoconvent ion alASCII. A hig h-order 110
indic ates thatthe runetakes 2 bytes; the secon d byte beg inswit h 10.Largerrunes haveanalo-
gous encodings.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
68 CHAPTER 3. BASIC DATATYPES
0xxxxxx runes0127 (ASCII)
11xxxxx 10xxxxxx 1282047 (values <128 unus ed)
110xxxx 10xxxxxx 10xxxxxx 204865535 (values <2048 unus ed)
1110xxx 10xxxxxx 10xxxxxx 10xxxxxx 655360x10f f f f (other values unus ed)
Avar iable-lengt h enco dingpre cludes direc t indexingtoaccessthe n-t h ch arac ter ofastr ing ,
butUTF-8 has manydesirable pro per ties tocompens ate. The encodingiscompact,compati-
blewit h ASCII, andself-sy nchro nizing: itspossibletofind the beg inningofacharac ter by
back ing upnomorethanthree bytes. Itsals o aprexcode, soitcan bedecoded fro m lef t to
rightwit houtany ambiguity orlookahead. Norunesencodingisasubst ringofany other,or
even ofa sequence ofothers, soyou can searc h forarunebyjustsearc hingfor itsbytes,
withoutwor rying about the pre cedingcontext. Thelexicog raphic byteorder equ als the Uni-
co de co de pointorder,sosor tingUTF-8 wor ksnatural ly. There are noemb edde d NUL (zero)
bytes, whichisconvenientfor prog ramminglangu ages thatuse NUL toter minatestr ings.
Go source files arealways enco dedinUTF-8, andUTF-8 isthe preferred encodingfor text
st rings manipu lated byGoprograms. The unicode packageprovides functions for wor king
with indiv idu alrunes (suchasdistinguishingletters fro m numb ers,orconvertinganupp er-
case lettertoalow er-case one), andthe unicode/utf8 packageprovides functions for encod-
ingand decodingrunes as bytes usingUTF-8.
Many Unico de ch arac tersare hardtotyp e on a key board ortodistinguish visuallyfro m sim-
ilar-lo oking ones; som e areeveninv isible. Unico de es cap es in Gostr ing lit eralsallow usto
sp ecif y them bytheir numer ic co de pointvalue.There are two for ms, \uhhhh fora16-bit
value and \Uhhhhhhhh fora32-bit value,where each h is a hexade cimal digit;the need for the
32-bit for m ar isesver y inf requently. Eachdenot esthe UTF-8 enco dingofthe specified code
point. Thus,for example, the fol low ing str ing lit eralsall represent the samesix-bytestr ing:
"BF"
"\xe4\xb8\x96\xe7\x95\x8c"
"\u4e16\u754c"
"\U00004e16\U0000754c"
Thethree escap e sequences abov e prov ide alternat ive not ation s forthe rs t st ring, but the val-
ues the y denot e areidentical.
Unico de es cap es mayals o be used in runelit erals. Thes e thre e literalsare equivalent:
'B''\u4e16''\U00004e16'
Arunewhose value islessthan256 may bewritt enwit h asinglehexade cimal escape,suchas
'\x41' for 'A',but for hig her values, a \u or \U es cap e mu stbeused. Con sequently,
'\xe4\xb8\x96' is not a legal runelit eral,eventhoug h thos e thre e bytesare a valid UTF-8
enco dingofasinglecodepoint.
Than ks to the nice pro per ties ofUTF-8, manystr ing operat ions dontrequiredecoding. We
cantestwhether one str ing containsanother as a prefix:
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 3.5. STRINGS 69
func HasPrefix(s, prefix string) bool {
return len(s) >= len(prefix) && s[:len(prefix)] == prefix
}
or asasuffix:
func HasSuffix(s, suffix string) bool {
return len(s) >= len(suffix) && s[len(s)-len(suffix):] == suffix
}
or asasubst ring:
func Contains(s, substr string) bool {
for i := 0; i < len(s); i++ {
if HasPrefix(s[i:], substr) {
return true
}
}
return false
}
usingthe samelog ic forUTF-8-enco dedtext as for raw bytes. Thisisnot truefor other
enco dings. (Thefunctions above are draw n from the strings package, thoug h itsimp lemen-
tation of Contains us esahashingtechnique tosearc h more efficiently.)
On the other hand, ifwereallycareabout the indiv idu alUnico de ch arac ters, wehavetouse
ot her mechanisms. Con sider the str ing fro m ourver y rs t example, whichincludes two East
Asiancharac ters. Figure3.5 illustrates its represent ation inmemor y.The str ing contains13
bytes, but int erprete d as UTF-8, itencodes onlyninecodepointsorrunes:
import "unicode/utf8"
s:="Hello, BF"
fmt.Println(len(s)) // "13"
fmt.Println(utf8.RuneCountInString(s)) // "9"
To pro cessthose charac ters, weneed a UTF-8 deco der.The unicode/utf8 packageprovides
on e that wecan use likethis:
for i := 0; i < len(s); {
r, size := utf8.DecodeRuneInString(s[i:])
fmt.Printf("%d\t%c\n", i, r)
i+=size
}
Each cal l to DecodeRuneInString returns r,the runeits elf,and size,the numberofbytes
occupied bythe UTF-8 enco dingof r.The size isusedtoupdatethe byteindex i of the next
rune inthe str ing . Butthisisclumsy, and weneed loops ofthiskindall the time. For tunately,
Gos range lo op, whenapp lie d to a str ing , perfor msUTF-8 deco dingimp licitly. The out put of
theloopbelow isals o shown inFigure3.5; not ice how the index jumps bymorethan1for
each non-ASCII rune.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
70 CHAPTER 3. BASIC DATATYPES
Figure 3.5. A range lo opdecodes a UTF-8-enco dedstr ing .
for i, r := range "Hello, BF"{
fmt.Printf("%d\t%q\t%d\n", i, r, r)
}
We cou lduse a simple range lo optocount the numberofrunes in a str ing , li kethis:
n:=0
for _, _ = range s {
n++
}
As wit h theother for msof range lo op, wecan omitthe var iables wedontneed:
n:=0
for range s {
n++
}
Or wecan justcal l utf8.RuneCountInString(s).
We mention edearlier thatitismostlyamatterofconvent ion inGothattext str ingsare int er-
preted asUTF-8-enco dedsequences ofUnico de co de points, but for cor rec t us e of range
lo ops onstr ings, itsmorethanaconvent ion,itsanecessity.Whathappens ifwerange overa
st ringcontainingarbit rar y binary dat a or,for thatmatter, UTF-8 dat a cont ainingerror s?
Each timeaUTF-8 deco der,whether explicitinacal l to utf8.DecodeRuneInString or
implicitinarange lo op, con sumes an unexp ected inp utbyte, itgenerates a speci al Unico de
re placement chara cter, '\uFFFD',whichisusu allyprint edasawhite question markinside a
bl ack hexagon alordiamond-li keshape (.Whenaprogram encounters thisrunevalue,its
of ten a signthatsom e upst reampar t of the systemthatgenerated the str ing dat a hasbeen
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 3.5. STRINGS 71
care lessinits tre atmentoftext enco dings.
UTF-8 isexception allyconvenient as an int erc hange for mat but wit hin a program runes may
be moreconvenientbecause the y areofunifor m size andare thu s easi lyindexed inarrays and
slices.
A []rune conv ersionapp lie d to a UTF-8-enco dedstr ing retur nsthe sequence ofUnico de
co de points thatthe str ing encodes:
// "program" in Japanese katakana
s:=">+=@?"
fmt.Printf("% x\n", s) // "e3 83 97 e3 83 ad e3 82 b0 e3 83 a9 e3 83 a0"
r:=[]rune(s)
fmt.Printf("%x\n", r) // "[30d7 30ed 30b0 30e9 30e0]"
(Theverb %xin the rs t Printf insertsaspace bet weeneachpair ofhex digits.)
If a slice ofrunes isconverted toastr ing , it produces the con catenat ionofthe UTF-8 enco d-
ings ofeachrune:
fmt.Println(string(r)) // ">+=@?"
Conv ertinganint egervalue toastr ing int erprets the int egerasarunevalue,and yieldsthe
UTF-8 represent ation of thatrune:
fmt.Println(string(65)) // "A", not "65"
fmt.Println(string(0x4eac)) // "C"
If the runeisinvalid,the replacementcharac ter issubst itute d:
fmt.Println(string(1234567)) // "("
3.5.4. Strings and ByteSlices
Fo urstand ard packages arepar tic ularlyimp ortantfor manipu lat ingstr ings: bytes, strings,
strconv,and unicode.The strings packageprovides manyfunctions for searc hing, replac-
ing, comparing , tr imming, split ting, and joiningstr ings.
The bytes packagehas similarfunctions for manipu lat ingslices ofbytes, oftyp e []byte,
whichshare som e prop erties wit h st rings. Because str ingsare immut able, bui ldingupstr ings
incrementallycan invo l ve alot ofallocat ionand copying . In suchcas es, itsmoreefficientto
us e the bytes.Buffer type,whichwell show in a mom ent.
The strconv packageprovides functions for convertingboole an, integer, and floating-p oint
values toand fro m their str ing represent ation s,and functions for quoting and unquoting
st rings.
The unicode packageprovides functions like IsDigit, IsLetter, IsUpper,and IsLower for
cl assif yingrunes. Eachfunctiontakes a singleruneargumentand retur nsaboole an. Conv er-
sionfunctions like ToUpper and ToLower conv ert a rune int o thegiven cas e if itisaletter. All
thes e func tions use the Unico de st and ard categor ies for letters,dig its, andsoon. The strings
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
72 CHAPTER 3. BASIC DATATYPES
packagehas similarfunctions,als o called ToUpper and ToLower,thatretur n anew str ing wit h
thespecified transfor mat ionapp lie d to eachcharac ter ofthe originalstr ing .
The basename func tionbelow was inspired bythe Unix shel l ut i lit y of the samename. Inour
version, basename(s) removesany prexof s that lookslikeafile systempat h with com-
ponentssep arated byslashes, anditremov esany suffixthatlooks like a file typ e:
fmt.Println(basename("a/b/c.go")) // "c"
fmt.Println(basename("c.d.go")) // "c.d"
fmt.Println(basename("abc")) // "abc"
The rs t versionof basename do es al l thewor k withoutthe helpoflibrar ies:
gopl.io/ch3/basename1
// basename removes directory components and a .suffix.
// e.g., a => a, a.go => a, a/b/c.go => c, a/b.c.go => b.c
func basename(s string) string {
// Discard last '/' and everything before.
for i := len(s) - 1; i >= 0; i-- {
if s[i] == '/' {
s=s[i+1:]
break
}
}
// Preserve everything before last '.'.
for i := len(s) - 1; i >= 0; i-- {
if s[i] == '.' {
s=s[:i]
break
}
}
return s
}
Asimpler versionusesthe strings.LastIndex librar y func tion:
gopl.io/ch3/basename2
func basename(s string) string {
slash := strings.LastIndex(s, "/") // -1 if "/" not found
s=s[slash+1:]
if dot := strings.LastIndex(s, "."); dot >= 0 {
s=s[:dot]
}
return s
}
The path and path/filepath packages provide a moregeneral set offunctions for manip-
ulat inghierarchic al names. The path packagewor kswit h sl ash-delimite d pathsonany plat-
form.Itshouldntbeusedfor file names, but it isappro priatefor other domains, like the pat h
comp onent ofaURL. Bycontrast, path/filepath manipu lates file names usingthe rules for
thehostplatfor m,suchas /foo/bar forPOSIX or c:\foo\bar on Micros oft Windows.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 3.5. STRINGS 73
Letscontinuewit h anot her subst ringexample. The taskistotakeastr ing represent ation of an
integer, suchas "12345",and ins ert commasevery three places, as in "12,345".Thisversion
on lywor ksfor int egers;handling floating-p ointnumbers islef t as a exercise.
gopl.io/ch3/comma
// comma inserts commas in a non-negative decimal integer string.
func comma(s string) string {
n:=len(s)
if n <= 3 {
return s
}
return comma(s[:n-3]) + "," + s[n-3:]
}
Theargumentto comma is a str ing . If its lengt h is lessthanorequ alto3,nocomma is neces-
sary.Other wis e, comma callsits elf rec ursivelywit h asubst ringcon sisting ofall but the last
thre e ch arac ters, andapp end s acomma and the lastthree charac terstothe resultofthe rec ur-
sive cal l.
Astr ing containsanarray ofbytes that, oncecre ate d, is immut able. Bycontrast, the elements
of a byteslice can befre ely modified.
St rings can beconverted tobyteslices andbackagain:
s:="abc"
b:=[]byte(s)
s2 := string(b)
Conceptu ally, the []byte(s) conv ersionallocates a new bytearray holdingacopyofthe bytes
of s,and yieldsaslice thatreferences the ent irety ofthatarray.Anopt imizingcompi ler may
be abletoavoid the allocat ionand copying insom e cases, but ingeneral copy ing isrequired to
ensure thatthe bytes of s remain unch ange d even if those of b aresubsequentlymodified.The
conv ersionfro m byte slice backtostr ing wit h string(b) also makes a copy,toens ure
immut abi lit y of the resulting str ing s2.
To avoid conv ersions and unnecessary memor y al location,manyofthe utilit y func tions inthe
bytes packagedirec tly paral leltheir counterpar ts in the strings package. For example, here
arehalf a dozen functions fro m strings:
func Contains(s, substr string) bool
func Count(s, sep string) int
func Fields(s string) []string
func HasPrefix(s, prefix string) bool
func Index(s, sep string) int
func Join(a []string, sep string) string
andthe cor respondingonesfro m bytes:
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
74 CHAPTER 3. BASIC DATATYPES
func Contains(b, subslice []byte) bool
func Count(s, sep []byte) int
func Fields(s []byte) [][]byte
func HasPrefix(s, prefix []byte) bool
func Index(s, sep []byte) int
func Join(s [][]byte, sep []byte) []byte
Theonlydif ference isthatstr ingshavebeenreplace d by byte slices.
The bytes packageprovides the Buffer type for efficientmanipu lat ionofbyteslices. A
Buffer st artsout emp tybut grows as dat a of typ es li ke string, byte,and []byte arewritt en
to it. Asthe examplebelow shows, a bytes.Buffer var iable requires noinitializat ionbecause
itszerovalue isusable:
gopl.io/ch3/printints
// intsToString is like fmt.Sprintf(values) but adds commas.
func intsToString(values []int) string {
var buf bytes.Buffer
buf.WriteByte('[')
for i, v := range values {
if i > 0 {
buf.WriteString(", ")
}
fmt.Fprintf(&buf, "%d", v)
}
buf.WriteByte(']')
return buf.String()
}
func main() {
fmt.Println(intsToString([]int{1, 2, 3})) // "[1, 2, 3]"
}
Wh enapp endingthe UTF-8 enco dingofanarbit rar y rune toabytes.Buffer,itsbesttouse
bytes.Buffers WriteRune method,but WriteByte is ne for ASCII charac terssuchas '['
and ']'.
The bytes.Buffer type isext remelyversatile, and whenwedis cussint erfaces in Chapt er7,
well see how itmay beusedasareplacementfor a file whene ver an I/O functionrequires a
sin k forbytes (io.Writer)as Fprintf do es ab ove , or a sourceofbytes (io.Reader).
Exercis e 3.10: Wr ite a non-rec ursiveversionof comma,using bytes.Buffer insteadofstr ing
conc atenation.
Exercis e 3.11: En hance comma so thatitdeals cor rec tly wit h floating-p ointnumbers and an
opt ion alsig n.
Exercis e 3.12: Wr ite a functionthatrep ortswhether two str ingsare anagramsofeachother,
that is, the y cont ain the sameletters inadif ferentorder.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 3.6. CONSTANTS75
3.5.5. Conversions between Strings and Numbers
In addition toconversions bet weenstr ings, runes, andbytes, itsoften necessary toconvert
betweennumer ic values andtheir str ing represent ation s.Thisisdon e with func tions fro m the
strconv package.
To convert anint egertoastr ing , on e opt ion istouse fmt.Sprintf;another istouse the func-
tion strconv.Itoa (‘‘integertoASCII’’):
x:=123
y:=fmt.Sprintf("%d", x)
fmt.Println(y, strconv.Itoa(x)) // "123 123"
FormatInt and FormatUint canbeusedtofor mat numbers inadif ferentbas e:
fmt.Println(strconv.FormatInt(int64(x), 2)) // "1111011"
The fmt.Printf verbs %b, %d, %u,and %x areoften moreconvenientthan Format func tions,
especi ally ifwewanttoinclude addition alinfor mat ionbesides the number:
s:=fmt.Sprintf("x=%b", x) // "x=1111011"
To parse a str ing represent ing anint eger, use the strconv func tions Atoi or ParseInt,or
ParseUint forunsig ned int egers:
x, err := strconv.Atoi("123") // x is an int
y, err := strconv.ParseInt("123", 10, 64) // base 10, up to 64 bits
Thethirdargumentof ParseInt givesthe size ofthe int egertyp e that the resultmustfitint o;
forexample, 16imp lies int16,and the speci al value of0imp lies int.Inany cas e,the typ e of
theresult y is always int64,whichyou can thenconvert toasmaller typ e.
Sometimes fmt.Scanf is usef ulfor parsinginp utthatcon sists oforderly mixtures ofstr ings
andnumbers all onasingleline, but it can beinflexible, esp eci ally whenhandlingincomplete
or irregu lar input.
3.6. Constants
Cons tants areexpressions whose value isknown tothe compi ler andwhose evaluation isguar-
ante e d to occ ur at comp ile time, not atrun time. The underly ing typ e of every con stant isa
basic typ e: boole an, st ring, ornumber.
A const de clarat iondefinesnamed values thatlooksyntactic ally likevar iables but whose
value iscon stant,whichpre vents accidental(or nefar ious)changesdur ingprogram exec ution.
Fo r inst ance, a con stant ismoreappro priatethanavar iable for a mat hemat ical cons tantlike
pi,since its value wontchange:
const pi = 3.14159 // approximately; math.Pi is a better approximation
As wit h var iables, a sequence ofcon stantscan appear in one declarat ion; thiswou ldbe
appropriatefor a gro upofrel ate d values:
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
76 CHAPTER 3. BASIC DATATYPES
const (
e=2.71828182845904523536028747135266249775724709369995957496696763
pi = 3.14159265358979323846264338327950288419716939937510582097494459
)
Many computation s on con stantscan becompletelyevaluate d at comp ile time, reducingthe
work necessary atrun timeand enablingother comp iler opt imizat ions.Error s ordinar ily
detec ted atrun timecan berep orted atcompi letimewhentheir operands are con stants, such
as integerdiv isionbyzero, str ing indexingout of bound s,and any floating-p ointoperat ion
that wou ldresult in a non-finite value.
Theresults ofall arithmetic, logic al,and comparisonoperat ions app lie d to con stant operands
arethems elves cons tants, as arethe results ofconversions and cal lstocer tain bui lt-in func-
tion s such as len, cap, real, imag, complex,and unsafe.Sizeof (§13.1).
Sincetheir values areknown tothe compi ler,con stant expressions may appear in typ es, sp ecif-
ic ally asthe lengt h of anarray typ e:
const IPv4Len = 4
// parseIPv4 parses an IPv4 address (d.d.d.d).
func parseIPv4(s string) IP {
var p [IPv4Len]byte
// ...
}
Acon stant declarat ionmay specif y atyp e as wel l as a value,but inthe abs ence ofanexplicit
type,the typ e is infer red fro m theexpressiononthe rig ht-handside.Inthe fol low ing ,
time.Duration is a name d type whose underly ing typ e is int64,and time.Minute is a con-
st ant ofthattyp e.Bot h of the con stantsdeclare d belowthu s have the typ e time.Duration as
we ll, asreveale d by %T:
const noDelay time.Duration = 0
const timeout = 5 * time.Minute
fmt.Printf("%T %[1]v\n", noDelay) // "time.Duration 0"
fmt.Printf("%T %[1]v\n", timeout) // "time.Duration 5m0s
fmt.Printf("%T %[1]v\n", time.Minute) // "time.Duration 1m0s"
Wh enasequence ofcon stantsisdeclare d as a gro up, the rig ht-handside expressionmay be
omit ted for all but the rs t of the gro up, imp l yingthatthe pre vious expressionand its typ e
shouldbeusedagain. For example:
const (
a=1
b
c=2
d
)
fmt.Println(a, b, c, d) // "1 1 2 2"
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 3.6. CONSTANTS77
Thisisnot ver y us efulifthe imp licitlycopie d right-handside expressionalways evaluatesto
thesamething. But what if it cou ldvar y? Thisbringsusto iota.
3.6.1. The ConstantGenerator iota
A const de clarat ionmay use the constant generat o r iota,whichisusedtocre ate a sequence
of rel ate d values wit houtspellingout eachone explicitly. Inaconst de clarat ion, the value of
iota begins atzeroand incrementsbyone for eachitem in the sequence.
Heresanexamplefro m the time package, whichdefinesnamed con stantsoftyp e Weekday for
thedays ofthe week, startingwit h zerofor Sunday.Typ es of thiskindare often cal le d enu-
merati ons,or enums forshort.
type Weekday int
const (
Sunday Weekday = iota
Monday
Tuesday
Wednesday
Thursday
Friday
Saturday
)
Thisdeclares Sunday to be0, Monday to be 1, and soon.
We can use iota in morecomplex expressions too,asinthisexamplefro m the net package
whereeachofthe low est 5 bitsofanunsig ned int egerisgiven a dist inc t name and boole an
interpretation:
type Flags uint
const (
FlagUp Flags = 1 << iota // is up
FlagBroadcast // supports broadcast access capability
FlagLoopback // is a loopback interface
FlagPointToPoint // belongs to a point-to-point link
FlagMulticast // supports multicast access capability
)
As iota increments, eachcon stant isassig ned the value of 1<<iota,whichevaluatestosuc-
cessive pow ers oftwo,eachcor respondingtoasinglebit.Wecan use these con stantswit hin
func tions thattest, set, orcle ar on e or moreofthese bits:
gopl.io/ch3/netflag
func IsUp(v Flags) bool {return v&FlagUp == FlagUp }
func TurnDown(v *Flags) {*v&^= FlagUp }
func SetBroadcast(v *Flags) { *v |= FlagBroadcast }
func IsCast(v Flags) bool {return v&(FlagBroadcast|FlagMulticast) != 0 }
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
78 CHAPTER 3. BASIC DATATYPES
func main() {
var v Flags = FlagMulticast | FlagUp
fmt.Printf("%b %t\n", v, IsUp(v)) // "10001 true"
TurnDown(&v)
fmt.Printf("%b %t\n", v, IsUp(v)) // "10000 false"
SetBroadcast(&v)
fmt.Printf("%b %t\n", v, IsUp(v)) // "10010 false"
fmt.Printf("%b %t\n", v, IsCast(v)) // "10010 true"
}
As a morecomplex exampleof iota,thisdeclarat ionnames the pow ers of1024:
const (
_=1<<(10 * iota)
KiB // 1024
MiB // 1048576
GiB // 1073741824
TiB // 1099511627776 (exceeds 1 << 32)
PiB // 1125899906842624
EiB // 1152921504606846976
ZiB // 1180591620717411303424 (exceeds 1 << 64)
YiB // 1208925819614629174706176
)
The iota me chanism has its limits. For example, itsnot possibletogeneratethe more fami l-
iarpow ers of1000 (KB,MB, and soon) because there isnoexp onent iat ionoperator.
Exercis e 3.13: Wr ite const de clarat ions for KB, MB, upthrough YB as compactly asyou can.
3.6.2. Untyped Constants
Cons tants in Goare a bit unu sual. Alt hough a con stant can haveany ofthe basic dat a types
li ke int or float64,includingnamed basic typ es li ke time.Duration,manycon stantsare
notcommitt edtoapar tic ulartyp e.The compi ler representsthese uncommitt edcon stants
with muchgre aternumer ic precisionthanvalues ofbasic typ es, andarithmeticonthemis
more pre cis e than machinearithmetic; you may assume atleast 256 bitsofpre cision. There
aresix flavor s of these uncommitt edcon stants, cal le d unty ped boole an, untypedint eger,
untypedrune, unt ypedfloating-p oint, untypedcomplex, andunt ypedstr ing .
By defer r ingthiscommitment, untypedcon stantsnot onlyret ain their higherpre cisionunt i l
later, but the y canpar ticipate inmanymoreexpressions thancommitt edcon stantswit hout
re quir ingconversions.For example, the values ZiB and YiB in the exampleabove are too big
to store inany int egervar iable,but the y areleg itimate con stantsthatmay beusedinexpres-
sions likethisone:
fmt.Println(YiB/ZiB) // "1024"
As another example, the floating-p ointcon stant math.Pi maybeusedwhere ver any floating-
pointorcomplex value isneeded:
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 3.6. CONSTANTS79
var x float32 = math.Pi
var y float64 = math.Pi
var z complex128 = math.Pi
If math.Pi hadbeencommitt edtoaspecifictyp e such as float64,the resultwou ldnot beas
precis e,and typ e conv ersions wouldberequired touse itwhenafloat32 or complex128
value iswante d:
const Pi64 float64 = math.Pi
var x float32 = float32(Pi64)
var y float64 = Pi64
var z complex128 = complex128(Pi64)
Fo r literals, synt axdeter mines flavor.The lit erals 0, 0.0, 0i,and '\u0000' al l denot e con-
st antsofthe samevalue but dif ferent flavor s:unt ypedint eger, unt ypedfloating-p oint, untyped
comp lex, andunt ypedrune, respectively. Simi larly, true and false areunt ypedboole ansand
st ringlit eralsare unt ypedstr ings.
Re call that / mayrepresent int egerorfloating-p ointdiv isiondep endingonits operands.
Cons equently, the choice oflit eral may affe ctthe resultofacon stant div isionexpression:
var f float64 = 212
fmt.Println((f - 32) * 5 / 9) // "100"; (f - 32) * 5 is a float64
fmt.Println(5 / 9 * (f - 32)) // "0"; 5/9 is an untyped integer, 0
fmt.Println(5.0 / 9.0 * (f - 32)) // "100"; 5.0/9.0 is an untyped float
On lycon stantscan beunt yped. When an unt ypedcon stant isassig ned toavar iable, as in the
rs t st atement below,orapp earsonthe rig ht-handside ofavar iable decl arat ionwit h an
explicittyp e, as in the other three statements, the con stant isimp licitlyconverted tothe typ e
of thatvar iable if possible.
var f float64 = 3 + 0i // untyped complex -> float64
f=2 // untyped integer -> float64
f=1e123 // untyped floating-point -> float64
f='a' // untyped rune -> float64
Thestatementsabove are thu s equivalenttothese:
var f float64 = float64(3 + 0i)
f=float64(2)
f=float64(1e123)
f=float64('a')
Wh ether implicitorexplicit, conv ertingacon stant fro m on e type toanother requires thatthe
targettyp e canrepresent the originalvalue.Roundingisallow edfor realand complex oat-
ing-p ointnumbers:
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
80 CHAPTER 3. BASIC DATATYPES
const (
deadbeef = 0xdeadbeef // untyped int with value 3735928559
a=uint32(deadbeef) // uint32 with value 3735928559
b=float32(deadbeef) // float32 with value 3735928576 (rounded up)
c=float64(deadbeef) // float64 with value 3735928559 (exact)
d=int32(deadbeef) // compile error: constant overflows int32
e=float64(1e309) // compile error: constant overflows float64
f=uint(-1) // compile error: constant underflows uint
)
In a var iable decl arat ionwit hout an explicittyp e (includingshort var iable decl arat ions), the
flavor of the unt ypedcon stant imp licitlydeter mines the defau lttyp e of the var iable,asinthese
examples:
i:=0 //untyped integer; implicit int(0)
r:='\000'// untyped rune; implicit rune('\000')
f:=0.0 // untyped floating-point; implicit float64(0.0)
c:=0i//untyped complex; implicit complex128(0i)
No tethe asy mmet ry: unt ypedint egers are converted to int,whose size isnot guarante e d, but
untypedfloating-p ointand complex numbers are converted tothe explicitlysize d types
float64 and complex128.The langu agehas nounsize d float and complex typesanalogou s
to unsize d int,because itisver y difficult towrite cor rec t numericalalgor it hms wit hout
know ing the size ofonesfloating-p ointdat a types.
To givethe var iable a different typ e,wemustexplicitlyconvert the unt ypedcon stant tothe
desired typ e or state the desired typ e in the var iable decl arat ion, as in these examples:
var i = int8(0)
var i int8 = 0
Thes e defau lts arepar tic ularlyimp ortantwhenconvertinganunt ypedcon stant toanint erface
value (see Chapter 7) since the y deter mineits dynamic typ e.
fmt.Printf("%T\n", 0) // "int"
fmt.Printf("%T\n", 0.0) // "float64"
fmt.Printf("%T\n", 0i) // "complex128"
fmt.Printf("%T\n", '\000')//"int32" (rune)
Weve now cov ere d thebasic dat a typesofGo. The next stepistoshowhow the y canbecom-
bine d into largergro upingslikearrays andstr ucts, andthenint o data str uctures for solv ing
re alprogrammingpro blems; thatisthe topic ofChapt er4.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
4
CompositeTypes
In Chapt er3 wedis cussedthe basic typ es that ser veasbui ldingblo cks for dat a st ruc tures in a
Go program; the y arethe atoms ofour univers e. In thischapt er, well takealookat comp osite
types, the mole cules create d by com biningthe basic typ es in various ways. Well tal k ab out
four suchtyp esar rays, slices, maps, andstr uctsandatthe end ofthe chapt er, well show
howstr uctured dat a usingthese typ es canbeencoded asand parsedfro m JSONdat a andused
to generateHTMLfro m temp lates.
Ar rays andstr uctsare ag gre gat e types; their values arecon catenat ions ofother values in mem-
or y.Arrays arehom ogene oustheir elements allhavethe sametyp ewhereasstr uctsare
heterogene ous.Bot h ar rays andstr uctsare xe d size.Incontrast, slices andmaps are
dy namic dat a st ruc tures thatgrow as values areadde d.
4.1. Arrays
An array isaxe d-lengt h sequence ofzeroormoreelements ofapar tic ulartyp e.Because of
their xe d lengt h, ar rays arerarelyuseddirec tly inGo. Slices, whichcan growand shrink,are
much moreversatile, but tounderst and slices wemustunderst and arrays rs t.
In div idu alarray elements areaccessedwit h theconvent ion alsubscript not ation,where
su bscriptsrun fro m zerotoone lessthanthe array lengt h. Thebui lt-in function len returns
thenumberofelements in the array.
var a [3]int // array of 3 integers
fmt.Println(a[0]) // print the first element
fmt.Println(a[len(a)-1]) // print the last element, a[2]
81
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
82 CHAPTER 4. COMPOSITE TYPES
// Print the indices and elements.
for i, v := range a {
fmt.Printf("%d %d\n", i, v)
}
// Print the elements only.
for _, v := range a {
fmt.Printf("%d\n", v)
}
By defau lt, the elements ofanew array variableare initial lyset tothe zerovalue for the ele-
ment typ e,whichis 0 fornumbers.Wecan use an ar ray lit era l to initialize an array wit h alist
of values:
var q [3]int = [3]int{1, 2, 3}
var r [3]int = [3]int{1, 2}
fmt.Println(r[2]) // "0"
In anarray literal,ifanellipsis ‘‘...’’ appearsinplace ofthe lengt h, thearray lengt h is deter-
mined bythe numberofinitializers. Thedefinition of q canbesimplified to
q:=[...]int{1, 2, 3}
fmt.Printf("%T\n", q) // "[3]int"
Thesize of an array ispar t of itstyp e,so [3]int and [4]int aredif ferenttyp es. Thesize
mu stbeacon stant expression, thatis, an expressionwhose value can becompute d as the
prog ram is beingcompi led.
q:=[3]int{1, 2, 3}
q=[4]int{1, 2, 3, 4} // compile error: cannot assign [4]int to [3]int
As well see,the lit eral synt axissimi lar for arrays, slices, maps, andstr ucts. Thespecificfor m
ab ove isalistofvalues in order,but it isals o possible tospecif y alistofindex andvalue pairs,
li kethis:
type Currency int
const (
USD Currency = iota
EUR
GBP
RMB
)
symbol := [...]string{USD: "$", EUR: "9", GBP: "!", RMB: """}
fmt.Println(RMB, symbol[RMB]) // "3 ""
In thisfor m,indices can appear in anyorder andsom e maybeomitt ed; as before, uns pecified
values takeonthe zerovalue for the elementtyp e.For ins tance,
r:=[...]int{99: -1}
defines an array r with 100 elements, allzeroexceptfor the last, whichhas value 1.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 4.1. ARRAYS83
If anarrayselementtyp e is comp arab le then the array typ e is comparable too,sowemay
direc tly compare two arrays ofthattyp e usingthe == op erator,whichrep ortswhether allcor-
resp ondingelements areequ al. The != op erator isits negat ion.
a:=[2]int{1, 2}
b:=[...]int{1, 2}
c:=[2]int{1, 3}
fmt.Println(a == b, a == c, b == c) // "true false false"
d:=[3]int{1, 2}
fmt.Println(a == d) // compile error: cannot compare [2]int == [3]int
As a moreplausible example, the function Sum256 in the crypto/sha256 packagepro duces
theSHA256 cryptographic hashor di gest of a message store d in an arbit rar y byte slice.The
digesthas 256 bits, soits typ e is [32]byte.Iftwo digests arethe same, itisext remelylikely
that the two messagesare the same; if the digests differ, the two messagesare dif ferent. This
prog ram pr intsand comparesthe SHA256 digests of "x" and "X":
gopl.io/ch4/sha256
import "crypto/sha256"
func main() {
c1 := sha256.Sum256([]byte("x"))
c2 := sha256.Sum256([]byte("X"))
fmt.Printf("%x\n%x\n%t\n%T\n", c1, c2, c1 == c2, c1)
// Output:
// 2d711642b726b04401627ca9fbac32f5c8530fb1903cc4db02258717921a4881
// 4b68ab3847feda7d6c62c1fbcbeebfa35eab7351ed5e78f4ddadea5df64b8015
// false
// [32]uint8
}
Thetwo inp uts differbyonlyasinglebit,but approximatelyhalf the bitsare dif ferentinthe
digests. Not ice the Printf verbs: %x to print all the elements ofanarray orslice ofbytes in
hexade cimal, %t to showaboole an, and %T to displ aythe typ e of a value.
Wh enafunctioniscal le d,acopyofeachargumentvalue isassig ned tothe cor responding
parameter variable, sothe functionreceivesacopy, not the original. Passinglarge arrays in
this way can beinefficient, andany changesthatthe functionmakes toarray elements affe ct
on lythe copy, not the original. Inthisregard, Gotre ats arrays like any other typ e,but this
behavior isdif ferentfro m languagesthatimp licitlypassarrays by reference.
Of course,wecan explicitlypassapoint ertoanarray sothatany modification s thefunction
makestoarray elements will bevisible tothe cal ler.Thisfunctionzeroesthe contentsofa
[32]byte ar ray:
func zero(ptr *[32]byte) {
for i := range ptr {
ptr[i] = 0
}
}
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
84 CHAPTER 4. COMPOSITE TYPES
Thearray literal [32]byte{} yields anarray of32bytes. Eachelementofthe array has the
zerovalue for byte,whichiszero. Wecan use thatfac t to write a dif ferentversionof zero:
func zero(ptr *[32]byte) {
*ptr = [32]byte{}
}
Usingapoint ertoanarray isefficientand allowsthe cal le d func tiontomut ate the cal lers
var iable,but arrays arestill inherentlyinflexiblebecause oftheir xe d size.The zero func tion
wi l l notacceptapoint ertoa[16]byte var iable,for example, nor isthere any way toadd or
remove array elements. For these reasons,other thanspeci al caseslikeSHA256sfixe d-size
hash,arrays areseldomused as functionparameters; instead, weuse slices.
Exercis e 4.1: Wr ite a functionthatcountsthe numberofbitsthatare dif ferent in two SHA256
hashes. (See PopCount from Sec tion 2.6.2.)
Exercis e 4.2: Wr ite a program thatprintsthe SHA256 hashofits stand ard inp utbydefau ltbut
supp ortsacommand-line flag toprint the SHA384 orSHA512 hashins tead.
4.2. Slices
Slices represent var iable-lengt h sequences whose elements allhavethe sametyp e.Aslice typ e
is writt en []T,where the elements havetyp e T;itlooks like anarray typ e withoutasize.
Ar rays andslices areint imate lyconne cte d. Aslice isalig htweig htdat a st ruc turethatgives
accesstoasubsequence (orperhaps all) ofthe elements ofanarray,whichisknown asthe
slices un derly ing array.Aslice has three components: a point er, a lengt h, andacap acity.The
pointerpointstothe rs t elementofthe array thatisreach ablethrough the slice,whichisnot
ne cessarily the arraysfirs t element. Thelengt h is the numberofslice elements; itcantexceed
thecap acity,whichisusu allythe numberofelements bet weenthe start ofthe slice andthe end
of the underly ing array.The bui lt-in functions len and cap return thos e values.
Mu ltipleslices can share the sameunderly ing array andmay refer tooverl appingpar ts of that
ar ray.Figure4.1 shows an array ofstr ingsfor the mont hsofthe year, and two overl apping
slices ofit. Thearray isdeclare d as
months := [...]string{1: "January", /* ... */, 12: "December"}
so January is months[1] andDecemberis months[12].Ordinar ily,the array elementatindex
0wou ldcontain the rs t value,but because monthsare always numbere d from 1,wecan leave
it out of the declarat ionand itwill beinitialize d to anemp tystr ing .
The sli ceoperator s[i:j],where 0 i j cap(s),cre atesanew slice thatreferstoelements
i thro ugh j-1 of the sequence s,whichmay beanarray variable, a point erto an array,or
anot her slice.The resulting slice has j-i elements. If i is omitt ed, its0,and if j is omitt ed, its
len(s).Thu s theslice months[1:13] refers to the whole range ofvalid months, as does the
slice months[1:];the slice months[:] refers to the whole array.Letsdefine overl appingslices
forthe secon d qu arter andthe northern summer :
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 4.2. SLICES 85
Figure 4.1. Tw o ov erl appingslices ofanarray ofmonths.
Q2 := months[4:7]
summer := months[6:9]
fmt.Println(Q2) // ["April" "May" "June"]
fmt.Println(summer) // ["June" "July" "August"]
June isinclude d in eachand isthe soleout put of this(inefficient) testfor commonelements:
for _, s := range summer {
for _, q := range Q2 {
if s == q {
fmt.Printf("%s appears in both\n", s)
}
}
}
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
86 CHAPTER 4. COMPOSITE TYPES
Slicingbeyon d cap(s) caus esapanic, but slicingbeyon d len(s) extends the slice,sothe
resu ltmay belon g erthanthe original:
fmt.Println(summer[:20]) // panic: out of range
endlessSummer := summer[:5] // extend a slice (within capacity)
fmt.Println(endlessSummer) // "[June July August September October]"
As anaside,not e thesimi lar ity ofthe subst ringoperat iononstr ingstothe slice operator on
[]byte slices. Bot h arewritt en x[m:n],and bot h return a subsequence ofthe originalbytes,
sh aring the underly ing represent ation sothatbot h op erat ions takecon stant time. The expres-
sion x[m:n] yields a str ing if x is a str ing , or a []byte if x is a []byte.
Sinceaslice cont ainsapoint ertoanelementofanarray,passingaslice toafunctionper mits
thefunctiontomodif y theunderly ing array elements. Inother words,copying a slice creates
an alias (§2.3.2) for the underly ing array.The function reverse re versesthe elements ofan
[]int slice in place,and itmay beapp lie d to slices ofany lengt h.
gopl.io/ch4/rev
// reverse reverses a slice of ints in place.
func reverse(s []int) {
for i, j := 0, len(s)-1; i < j; i, j = i+1, j-1 {
s[i], s[j] = s[j], s[i]
}
}
Here wereverse the whole array a:
a:=[...]int{0, 1, 2, 3, 4, 5}
reverse(a[:])
fmt.Println(a) // "[5 4 3 2 1 0]"
Asimpleway to ro tat e aslice leftby n elements istoapp l y the reverse func tionthree times,
rs t to the leading n elements, thentothe remainingelements, and nallytothe whole slice.
(Torot ate to the rig ht, makethe thirdcal l rs t.)
s:=[]int{0, 1, 2, 3, 4, 5}
// Rotate s left by two positions.
reverse(s[:2])
reverse(s[2:])
reverse(s)
fmt.Println(s) // "[2 3 4 5 0 1]"
No tice how the expressionthatinitializes the slice s dif fersfro m that for the array a.Asli ce
litera l lo oks like anarray literal,asequence ofvalues sep arated bycommasand sur rounded by
braces, but the size isnot given. Thisimp licitlycre atesanarray variableofthe rig htsize and
yields a slice thatpointstoit. Aswit h ar ray literals, slice literalsmay specif y thevalues in
order,orgivetheir indices explicitly, oruse a mix ofthe two sty les.
Un likearrays, slices arenot comparable,sowecannot use == to test whether two slices cont ain
thesameelements. Thestand ard librar y prov ides the hig hly opt imize d bytes.Equal func tion
forcomparing two slices ofbytes ([]byte), but for other typ es of slice,wemustdothe
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 4.2. SLICES 87
comp arisonourselves:
func equal(x, y []string) bool {
if len(x) != len(y) {
return false
}
for i := range x {
if x[i] != y[i] {
return false
}
}
return true
}
Gi venhow natural this ‘‘de ep’’ equality testis, andthatitisnomorecostlyatrun timethanthe
== op erator for arrays ofstr ings, itmay bepuzzlingthatslice comp arisons donot als o work
this way.There are two reasons why deepequivalence ispro blemat ic. First,unlikearray ele-
ments, the elements ofaslice areindirec t,mak ingitpossiblefor a slice tocontain its elf.
Although there are ways todealwit h such cas es, none issimple, efficient, andmost
importantly, obv iou s.
Second,because slice elements areindirec t,axe d slice value may cont ain different elements
at different times as the contentsofthe underly ing array aremodified.Because a hashtable
such asGosmap typ e makesonlyshallow copies ofits keys, itrequires thatequ ality for each
ke y remain the samethroughoutthe lifet imeofthe hashtable.Deepequivalence wou ldthu s
make slices unsuit ablefor use asmap keys. For reference typ es li kepoint ers and channel s,the
== op erator tests reference ident ity,thatis, whether the two ent ities refer tothe samething. An
analogou s ‘‘sh allow’’ equality testfor slices couldbeuseful, and itwou ldsolvethe pro blem
with maps, but the incon sistent tre atmentofslices andarrays bythe == op erator wou ldbe
conf using. The safestchoice istodis allow slice comp arisons alt ogether.
Theonlylegal slice comp arisonisagainst nil, as in
if summer == nil { /* ... */ }
Thezerovalue ofaslice typ e is nil.Ani l slice has nounderly ing array.The nil slice has
lengt h andcap acity zero, but there are als o non-ni l slices oflengt h andcap acity zero, suchas
[]int{} or make([]int, 3)[3:].Aswit h anytyp e that can havenil values, the nil value ofa
partic ularslice typ e canbewritt enusingaconversionexpressionsuchas []int(nil).
var s []int // len(s) == 0, s == nil
s=nil // len(s) == 0, s == nil
s=[]int(nil) // len(s) == 0, s == nil
s=[]int{} // len(s) == 0, s != nil
So,ifyou need totestwhether a slice isemp ty, use len(s) == 0,not s==nil.Other than
comp aring equ alto nil,anil slice beh aveslikeany other zero-lengt h slice; reverse(nil) is
perfec tly safe, for example. Unlesscle arly documente d to the contrar y,Gofunctions should
treatall zero-lengt h slices the sameway,whether nilornon-ni l.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
88 CHAPTER 4. COMPOSITE TYPES
Thebui lt-in function make createsaslice ofaspecified elementtyp e,lengt h, andcap acity.The
capacity argumentmay beomitt ed, inwhichcas e thecap acity equ als the lengt h.
make([]T, len)
make([]T, len, cap) // same as make([]T, cap)[:len]
Un der the hood, make createsanunnamed array variableand retur nsaslice ofit; the array is
accessibleonlythrough the retur ned slice.Inthe rs t form,the slice isavie w of the ent ire
ar ray.Inthe secon d,the slice isavie w of only the arraysfirs t len elements, but itscap acity
includes the ent ire array.The addition alelements areset aside for fut ure growt h.
4.2.1. The append Function
Thebui lt-in append func tionapp end s it ems toslices:
var runes []rune
for _, r := range "Hello, BF"{
runes = append(runes, r)
}
fmt.Printf("%q\n", runes) // "['H''e''l''l''o'',''''B''F']"
Theloopuses append to bui ld theslice ofninerunes enco dedbythe str ing lit eral,alt hough
this specificpro blem ismoreconvenientlysolved byusingthe bui lt-in conv ersion
[]rune("Hello, BF").
The append func tioniscruci al to underst andinghow slices wor k,soletstakealookatwhatis
goingon. Heresaversioncal le d appendInt that isspeci alizedfor []int slices:
gopl.io/ch4/append
func appendInt(x []int, y int) []int {
var z []int
zlen := len(x) + 1
if zlen <= cap(x) {
// There is room to grow. Extend the slice.
z=x[:zlen]
}else {
// There is insufficient space. Allocate a new array.
// Grow by doubling, for amortized linear complexity.
zcap := zlen
if zcap < 2*len(x) {
zcap = 2 * len(x)
}
z=make([]int, zlen, zcap)
copy(z, x) // a built-in function; see text
}
z[len(x)] = y
return z
}
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 4.2. SLICES 89
Each cal l to appendInt mu stche ckwhether the slice has sufficientcap acity toholdthe new
elements in the exist ing array.Ifso, itextends the slice bydefiningalargerslice (still wit hin
theoriginalarray), copies the element y into the newspace,and retur nsthe slice.The inp ut x
andthe result z sh are the sameunderly ing array.
If there isins ufficientspace for growt h, appendInt mu stallocateanew array big enough to
hold the result, copy the values fro m x into it, thenapp end the newelement y.The result z
nowreferstoadif ferentunderly ing array thanthe array that x refers to.
It wou ldbestraig htfor wardtocopythe elements wit h explicitloops, but itseasier touse the
built-in function copy,whichcopies elements fro m on e slice toanother ofthe sametyp e.Its
rs t argumentisthe destination and its secon d is the source, res emblingthe order ofoperands
in an assig nmentlike dst = src.The slices may refer tothe sameunderly ing array ; they may
even overl ap. Alt hough wedontuse ithere, copy returnsthe numberofelements actu ally
copied,whichisthe smaller ofthe two slice lengt hs, sothere isnodangerofrunningoff the
endoroverwriting som ethingout of range .
Fo r efficiency,the newarray isusu allysom ewhat largerthanthe minimum needed tohold x
and y.Exp andingthe array bydou blingits size at eachexp ansionavoidsanexcessive number
of allocat ions and ens uresthatapp endingasingleelementtakes cons tanttimeonaverage.
Thisprogram demon strates the effec t:
func main() {
var x, y []int
for i := 0; i < 10; i++ {
y=appendInt(x, i)
fmt.Printf("%d cap=%d\t%v\n", i, cap(y), y)
x=y
}
}
Each change incap acity indic ates an allo cat ionand a copy:
0cap=1 [0]
1cap=2 [0 1]
2cap=4 [0 12]
3cap=4 [0 123]
4cap=8 [0 1234]
5cap=8 [0 12345]
6cap=8 [0 123456]
7cap=8 [0 1234567]
8cap=16 [0 12345678]
9cap=16 [0 123456789]
Letstakeaclos er lo okatthe i=3 it erat ion. Theslice x cont ainsthe three elements [0 1 2] but
hascap acity 4,sothere isasingleelementofslack atthe end,and appendInt of the element 3
maypro ceed wit houtreallocat ing. The resulting slice y haslengt h andcap acity 4,and has the
same underly ing array as the originalslice x, as Figure4.2 shows.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
90 CHAPTER 4. COMPOSITE TYPES
Figure 4.2. Ap pendingwit h ro omtogrow.
On the next iterat ion, i=4,there isnoslack atall,so appendInt al locatesanew array ofsize 8,
copies the fourelements [0 1 2 3] of x,and app end s 4,the value of i.The resulting slice y
hasalengt h of 5 but a cap acity of8;the slackof3will savethe next three iterat ions fro m the
ne e d to reallocate. The slices y and x arevie ws of dif ferentarrays. Thisoperat ionisdepic ted
in Figure4.3.
Figure 4.3. Ap pendingwit houtroomtogrow.
Thebui lt-in append func tionmay use a moresop histicate d growth strateg y than appendInts
simplisticone.Usu allywedontknowwhether a given cal l to append wi l l caus e areallocat ion,
so wecantassume thatthe originalslice referstothe samearray as the resulting slice,nor that
it referstoadif ferentone.Simi larly,wemustnot assume thatoperat ions onelements ofthe
oldslice will (or will not)bereflec ted inthe newslice.Asaresult, itsusu altoassig n theresult
of a cal l to append to the sameslice variablewhose value wepassedto append:
runes = append(runes, r)
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 4.2. SLICES 91
Up dat ingthe slice variableisrequired not justwhencal ling append,but for any functionthat
maychange the lengt h or cap acity ofaslice ormakeitrefer toadif ferentunderly ing array.To
us e slices correc tly,itsimp ortanttobearinmindthatalt hough the elements ofthe underly ing
ar ray areindirec t,the slicespoint er, lengt h, andcap acity are not.Toupdatethemrequires an
assig nmentlikethe one above . In thisrespect,slices arenot ‘‘pure’’ referencetyp es butres em-
ble an aggregatetyp e such asthisstr uct:
type IntSlice struct {
ptr *int
len, cap int
}
Our appendInt func tionaddsasingleelementtoaslice,but the bui lt-in append lets usadd
more thanone newelement, orevenawhole slice ofthem.
var x []int
x=append(x, 1)
x=append(x, 2, 3)
x=append(x, 4, 5, 6)
x=append(x, x...) // append the slice x
fmt.Println(x) // "[1 2 3 4 5 6 1 2 3 4 5 6]"
Wi ththe small modification shown below,wecan match the beh avior of the bui lt-in append.
Theellipsis ‘‘...’’ in the declarat ionof appendInt makesthe function variadic:itaccepts any
numb eroffinalarguments. Thecor respondingellipsisinthe cal l ab ove to append shows how
to sup ply a listofarguments fro m aslice.Well explain thismechanism in detai l in
Section5.7.
func appendInt(x []int, y ...int) []int {
var z []int
zlen := len(x) + len(y)
// ...expand z to at least zlen...
copy(z[len(x):], y)
return z
}
Thelog ic to exp and zsunderly ing array remainsunchange d andisnot shown.
4.2.2. In-PlaceSliceTechniques
Letssee moreexamples offunctions that, like rotate and reverse,modif y theelements ofa
slice in place.Given a list ofstr ings, the nonempty func tionretur nsthe non-empt y on es:
gopl.io/ch4/nonempty
// Nonempty is an example of an in-place slice algorithm.
package main
import "fmt"
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
92 CHAPTER 4. COMPOSITE TYPES
// nonempty returns a slice holding only the non-empty strings.
// The underlying array is modified during the call.
func nonempty(strings []string) []string {
i:=0
for _, s := range strings {
if s != "" {
strings[i] = s
i++
}
}
return strings[:i]
}
Thesubtlepar t is thatthe inp utslice andthe out put slice share the sameunderly ing array.
Thisavoidsthe need toallocateanother array,thoug h of cours e thecontentsof data arepar tly
ov erwritt en, as evidence d by the secon d pr int statement:
data := []string{"one", "", "three"}
fmt.Printf("%q\n", nonempty(data)) // `["one" "three"]`
fmt.Printf("%q\n", data) // `["one" "three" "three"]`
Thus wewou ldusu allywrite: data = nonempty(data).
The nonempty func tioncan also bewritt enusing append:
func nonempty2(strings []string) []string {
out := strings[:0] // zero-length slice of original
for _, s := range strings {
if s != "" {
out = append(out, s)
}
}
return out
}
Whiche ver variant weuse,reusinganarray in thisway requires thatatmostone out put value
is pro duce d foreachinp utvalue,whichistrueofmanyalgor it hms thatfilt erout elements ofa
sequence orcom bine adj acentones. Suchint ric ateslice usage isthe exception,not the rule,
butitcan becle ar,efficient, andusefulonocc asion.
Aslice can beusedtoimp lementastack.Given an initial lyemp tyslice stack,wecan pusha
ne w value ont o theend ofthe slice wit h append:
stack = append(stack, v) // push v
Thetop ofthe stack isthe lastelement:
top := stack[len(stack)-1] // top of stack
andshr inking the stack bypoppingthatelementis
stack = stack[:len(stack)-1] // pop
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 4.3. MAPS 93
To remov e an elementfro m themidd le of a slice,preservingthe order ofthe remainingele-
ments, use copy to slide the hig her-numbere d elements down byone tofill the gap:
func remove(slice []int, i int) []int {
copy(slice[i:], slice[i+1:])
return slice[:len(slice)-1]
}
func main() {
s:=[]int{5, 6, 7, 8, 9}
fmt.Println(remove(s, 2)) // "[5 6 8 9]"
}
And if we dontneed topreservethe order,wecan justmov e thelastelementint o thegap:
func remove(slice []int, i int) []int {
slice[i] = slice[len(slice)-1]
return slice[:len(slice)-1]
}
func main() {
s:=[]int{5, 6, 7, 8, 9}
fmt.Println(remove(s, 2)) // "[5 6 9 8]
}
Exercis e 4.3: Re writ e reverse to use anarray point erins teadofaslice.
Exercis e 4.4: Wr ite a versionof rotate that operates in a singlepass.
Exercis e 4.5: Wr ite anin-place functiontoeliminateadj acentdup lic ates in a []string slice.
Exercis e 4.6: Wr ite anin-place functionthatsqu asheseachrun ofadj acentUnico de sp aces
(s ee unicode.IsSpace) in a UTF-8-enco ded []byte slice into a singleASCII space.
Exercis e 4.7: Mo dif y reverse to reverse the charac tersofa[]byte slice thatrepresentsa
UTF-8-enco dedstr ing , in place.Can you doitwit houtallocat ingnew memor y?
4.3. Maps
Thehashtable isone ofthe most ingenious and versatileofall dat a st ruc tures. Itisan
unordered col lec tion of key/value pairsinwhichall the keysare distinc t,and the value asso ci-
ated wit h agiven key can beret rie ved,updated,orremov edusingacon stant numb erofkey
comp arisons onthe average, nomatterhow large the hashtable.
In Go, a map is a reference toahashtable,and a map typ e is writt en map[K]V,where K and V
arethe typ es of itskeysand values. All ofthe keys in a given map areofthe sametyp e,and all
of the values areofthe sametyp e,but the keysneed not beofthe sametyp e as the values. The
ke y type K mu stbecomparable using ==,sothatthe map can testwhether a given key isequ al
to one already wit hin it. Though floating-p ointnumbers are comparable,itsabad ideato
comp are oats for equ ality and,aswemention edinChapt er3,esp eci ally bad if NaN isapos-
siblevalue.There are norestr ictions onthe value typ e V.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
94 CHAPTER 4. COMPOSITE TYPES
Thebui lt-in function make canbeusedtocre ate a map:
ages := make(map[string]int) // mapping from strings to ints
We can also use a maplit era l to cre ate a new map popu lated wit h some initial key/value pairs:
ages := map[string]int{
"alice": 31,
"charlie": 34,
}
Thisisequivalentto
ages := make(map[string]int)
ages["alice"] = 31
ages["charlie"] = 34
so analt ernat ive expressionfor a new emp tymap is map[string]int{}.
Mapelements areaccessedthrough the usu alsubscript not ation:
ages["alice"] = 32
fmt.Println(ages["alice"]) // "32"
andremov edwit h thebui lt-in function delete:
delete(ages, "alice") // remove element ages["alice"]
Al l of these operat ions are safeevenifthe elementisntinthe map; a map lookup usingakey
that isntpresent retur nsthe zerovalue for itstyp e,so, for ins tance,the fol low ing workseven
when "bob" is not yet a key inthe map because the value of ages["bob"] wi l l be 0.
ages["bob"] = ages["bob"] + 1 // happy birthday!
Theshorthand assig nmentfor ms x+=yand x++ also wor k formap elements, sowecan re-
wr ite the statement above as
ages["bob"] += 1
or evenmorecon cis ely as
ages["bob"]++
Butamap elementisnot a var iable,and wecannot takeits address:
_=&ages["bob"] // compile error: cannot take address of map element
Onereasonthatwecanttakethe addressofamap elementisthatgrowing a map might cause
re hashingofexist ing elements into new storagelocat ions,thu s potent ial lyinvalid atingthe
address.
To e numerateall the key/value pairsinthe map,weuse a range-b ased for lo opsimi lar to
thos e we saw for slices. Successive iterat ions ofthe loopcause the name and age var iables to
be set tothe next key/value pair :
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 4.3. MAPS 95
for name, age := range ages {
fmt.Printf("%s\t%d\n", name, age)
}
Theorder ofmap iterat ionisuns pecified,and dif ferentimp lementation s mig htuse a dif ferent
hash function, leadingtoadif ferentorder ing. Inprac tice,the order israndom, varying fro m
on e exec ution tothe next. Thisisint ent ion al; mak ingthe sequence varyhelps force programs
to berobustacrossimp lementation s.Toenumeratethe key/value pairsinorder,wemustsor t
thekeysexplicitly, for ins tance,usingthe Strings func tionfro m the sort package if the keys
arestr ings. Thisisacommonpattern:
import "sort"
var names []string
for name := range ages {
names = append(names, name)
}
sort.Strings(names)
for _, name := range names {
fmt.Printf("%s\t%d\n", name, ages[name])
}
Sinceweknowthe nalsize of names from the outset, itismoreefficienttoallocateanarray of
therequired size upfro nt. Thestatement below cre atesaslice thatisinitial lyemp tybut has
sufficientcap acity toholdall the keysofthe ages map:
names := make([]string, 0, len(ages))
In the rs t range lo opabove , we requireonlythe keysofthe ages map, soweomitthe secon d
lo opvar iable.Inthe secon d lo op, werequireonlythe elements ofthe names slice,soweuse
theblank identifier _ to ignorethe rs t var iable,the index.
Thezerovalue for a map typ e is nil,thatis, a reference tonohashtable at all.
var ages map[string]int
fmt.Println(ages == nil) // "true"
fmt.Println(len(ages) == 0) // "true"
Most operat ions onmaps, includinglooku p, delete, len,and range lo ops, aresafetoper-
form onanil map reference,since itbeh aveslikeanemp tymap.But storing toanil map
caus esapanic:
ages["carol"] = 21 // panic: assignment to entry in nil map
Yo u mu stallocatethe map beforeyou can store int o it.
Accessingamap elementbysubscript ing always yieldsavalue.Ifthe key ispresent inthe
map, you get the cor respondingvalue; if not,you get the zerovalue for the elementtyp e,aswe
sawwit h ages["bob"].For manypur pos es thatsfine,but som etimesyou need toknow
whet her the elementwas reallythere ornot.For example, ifthe elementtyp e is numer ic, you
mig hthavetodistinguish bet weenanon existentelementand anelementthathappens tohave
thevalue zero, usingatestlikethis:
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
96 CHAPTER 4. COMPOSITE TYPES
age, ok := ages["bob"]
if !ok { /* "bob" is not a key in this map; age == 0. */ }
Yo u ll often see these two statementscom bine d,likethis:
if age, ok := ages["bob"]; !ok { /* ... */ }
Su bscript ing a map in thiscontext yieldstwo values; the secon d is a boole an that rep orts
whet her the elementwas present.The boole an var iable isoften cal le d ok,esp eci ally ifitis
immediate lyused in an if condit ion.
As wit h slices, maps cannot becompare d to eachother ; theonlylegal comp arisoniswit h nil.
To testwhether two maps cont ain the samekeysand the sameass oci ated values, wemust
wr ite a loop:
func equal(x, y map[string]int) bool {
if len(x) != len(y) {
return false
}
for k, xv := range x {
if yv, ok := y[k]; !ok || yv != xv {
return false
}
}
return true
}
Observehow weuse !ok to distinguish the ‘‘missing’’ and ‘‘pres entbut zero’’ cases. Had we
naïvelywritt en xv != y[k],the cal l belowwou ldincor rec tly rep ort its arguments as equ al:
// True if equal is written incorrectly.
equal(map[string]int{"A": 0}, map[string]int{"B": 42})
Go doesnot provide a set type,but since the keysofamap aredistinc t,amap can ser vethis
purpos e.Toillustrate, the program dedup re ads a sequence oflines andprintsonlythe rs t
occurrence ofeachdistinc t line. (Itsavar iantofthe dup prog ram that weshowe d in
Section1.3.) The dedup prog ram us esamap whose keysrepresent the set oflines thathave
already app eared toens ure thatsubsequentocc urrences arenot print ed.
gopl.io/ch4/dedup
func main() {
seen := make(map[string]bool) // a set of strings
input := bufio.NewScanner(os.Stdin)
for input.Scan() {
line := input.Text()
if !seen[line] {
seen[line] = true
fmt.Println(line)
}
}
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 4.3. MAPS 97
if err := input.Err(); err != nil {
fmt.Fprintf(os.Stderr, "dedup: %v\n", err)
os.Exit(1)
}
}
Go programmersoften descr ibe a map usedinthisfashionasa‘‘setofstr ings’’ withoutfur ther
ado,but beware, not all map[string]bool values aresimplesets; som e maycontain bot h true
and false values.
Sometimesweneed a map orset whose keysare slices, but because a mapskeysmustbecom-
parable,thiscannot beexpress eddirec tly.How ever, itcan bedon e in two steps. First we
define a helperfunction k that maps eachkey toastr ing , with thepro per tythat k(x) == k(y)
if andonlyifwecon sider x and y equivalent. Then wecre ate a map whose keysare str ings,
apply ing the helperfunctiontoeachkey beforeweaccessthe map.
Theexamplebelow usesamap torecordthe numberoftimes Add hasbeencal le d with a given
list ofstr ings. Ituses fmt.Sprintf to convert a slice ofstr ingsint o asinglestr ing thatisa
suit ablemap key,quoting eachslice elementwit h %q to recordstr ing bound aries faithfully:
var m = make(map[string]int)
func k(list []string) string { return fmt.Sprintf("%q", list) }
func Add(list []string) {m[k(list)]++ }
func Count(list []string) int { return m[k(list)] }
Thesameappro ach can beusedfor any non-comp arable key typ e,not justslices. Itseven
us efulfor comparable key typ es when you wantadefinition of equ ality other than ==,suchas
case-insensit ive comparisons for str ings. And the typ e of k(x) ne e dntbeastr ing;any com-
parable typ e with thedesired equivalence pro per tywill do, such as int egers,arrays, orstr ucts.
Heresanother exampleofmaps in action,aprogram thatcountsthe occ urrences ofeachdis-
tinc t Unico de co de point in its input. Since there are a large numberofpossiblecharac ters,
on lyasmall frac tionofwhichwou ldapp ear in anypar tic ulardocument, a map isanatural
way tokeeptrackofjustthe onesthathavebeenseenand their correspondingcounts.
gopl.io/ch4/charcount
// Charcount computes counts of Unicode characters.
package main
import (
"bufio"
"fmt"
"io"
"os"
"unicode"
"unicode/utf8"
)
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
98 CHAPTER 4. COMPOSITE TYPES
func main() {
counts := make(map[rune]int) // counts of Unicode characters
var utflen [utf8.UTFMax + 1]int // count of lengths of UTF-8 encodings
invalid := 0 // count of invalid UTF-8 characters
in := bufio.NewReader(os.Stdin)
for {
r, n, err := in.ReadRune() // returns rune, nbytes, error
if err == io.EOF {
break
}
if err != nil {
fmt.Fprintf(os.Stderr, "charcount: %v\n", err)
os.Exit(1)
}
if r == unicode.ReplacementChar && n == 1 {
invalid++
continue
}
counts[r]++
utflen[n]++
}
fmt.Printf("rune\tcount\n")
for c, n := range counts {
fmt.Printf("%q\t%d\n", c, n)
}
fmt.Print("\nlen\tcount\n")
for i, n := range utflen {
if i > 0 {
fmt.Printf("%d\t%d\n", i, n)
}
}
if invalid > 0 {
fmt.Printf("\n%d invalid UTF-8 characters\n", invalid)
}
}
The ReadRune method perfor msUTF-8 deco dingand retur nsthree values: the decoded rune,
thelengt h in bytes ofits UTF-8 enco ding, and anerror value.The onlyerror weexp ect isend-
of-file.Ifthe inp utwas not a legal UTF-8 enco dingofarune, the retur ned runeis uni-
code.ReplacementChar andthe lengt h is 1.
The charcount prog ram also printsacount ofthe lengt hsofthe UTF-8 enco dings ofthe
runesthatapp eared inthe inp ut. A mapisnot the bestdat a st ruc turefor that; since enco ding
lengt hsrange onlyfro m 1to utf8.UTFMax (w hichhas the value 4), an array ismorecompact.
As anexp eriment, weran charcount on thisbookits elf at one point.Alt hough itsmostlyin
English, ofcours e, it doeshaveafair numberofnon-ASCII charac ters. Hereare the top ten:
°27 B 15 F 14 é 13 A 10 < 5 & 5 D 4 ( 4 + 3
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 4.4. STRUCTS99
andhereisthe distr ibution of the lengt hsofall the UTF-8 enco dings:
len count
1765391
260
370
40
Thevalue typ e of a map can its elf beacomposite typ e,suchasamap orslice.Inthe fol low ing
co de,the key typ e of graph is string andthe value typ e is map[string]bool,represent ing a
setofstr ings. Con ceptu ally, graph maps a str ing toaset ofrel ate d st rings, its successors ina
direc ted graph.
gopl.io/ch4/graph
var graph = make(map[string]map[string]bool)
func addEdge(from, to string) {
edges := graph[from]
if edges == nil {
edges = make(map[string]bool)
graph[from] = edges
}
edges[to] = true
}
func hasEdge(from, to string) bool {
return graph[from][to]
}
The addEdge func tionshows the idiomaticway topopu lateamap lazi ly, thatis, toinitialize
each value as its key app earsfor the rs t time.The hasEdge func tionshows how the zero
value ofamissingmap entr y is often put towor k:evenifneither from nor to is present,
graph[from][to] wi l l always giveameaningf ul resu lt.
Exercis e 4.8: Mo dif y charcount to count letters,dig its, andsoonintheir Unico de categories,
usingfunctions like unicode.IsLetter.
Exercis e 4.9: Wr ite a program wordfreq to rep ort the fre quency ofeachword in an inp uttext
file.Cal l input.Split(bufio.ScanWords) before the rs t call to Scan to bre akthe inp utint o
word s insteadoflines.
4.4. Structs
A st ruc t is anaggregatedat a type thatgro ups toget her zeroormorenamed values ofarbit rar y
typesasa singleent ity.Eachvalue iscal le d a el d.The classic exampleofastr uct fro m data
processingisthe emp loy eerecord, whose elds are a unique ID,the emp loy eesname, address,
date ofbir th, position,sal ary, manager,and the like. All ofthese elds are col lec ted int o asin-
gleent ity thatcan becopie d as a unit, passedtofunctions and retur ned bythem, store d in
ar rays, andsoon.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
100 CHAPTER 4. COMPOSITE TYPES
Thes e twostatementsdeclare a str uct typ e called Employee andavar iable cal le d dilbert that
is anins tance ofan Employee:
type Employee struct {
ID int
Name string
Address string
DoB time.Time
Position string
Salary int
ManagerID int
}
var dilbert Employee
Theindiv idu alfields of dilbert areaccessedusingdot not ation like dilbert.Name and
dilbert.DoB.Because dilbert is a var iable,its elds are var iables too,sowemay assig n to a
eld:
dilbert.Salary -= 5000 // demoted, for writing too few lines of code
or takeits addressand accessitthrough a point er:
position := &dilbert.Position
*position = "Senior " + *position // promoted, for outsourcing to Elbonia
Thedot not ation als o workswit h apoint ertoastr uct:
var employeeOfTheMonth *Employee = &dilbert
employeeOfTheMonth.Position += " (proactive team player)"
Thelaststatement isequivalentto
(*employeeOfTheMonth).Position += " (proactive team player)"
Gi venanemp loy eesunique ID,the function EmployeeByID returnsapoint ertoan Employee
st ruc t. We can use the dot not ation toaccessits elds:
func EmployeeByID(id int) *Employee { /* ... */ }
fmt.Println(EmployeeByID(dilbert.ManagerID).Position) // "Pointy-haired boss"
id := dilbert.ID
EmployeeByID(id).Salary = 0 // fired for... no real reason
Thelaststatement updates the Employee st ruc t that ispoint edtobythe resultofthe cal l to
EmployeeByID.Ifthe resulttyp e of EmployeeByID were change d to Employee insteadof
*Employee,the assig nmentstatement wou ldnot compi lesince its left-handside wou ldnot
identify a var iable.
Fieldsare usu allywritt enone per line, wit h thefieldsnamepre cedingits typ e,but con sec utive
elds ofthe sametyp e maybecom bine d, as wit h Name and Address here:
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 4.4. STRUCTS101
type Employee struct {
ID int
Name, Address string
DoB time.Time
Position string
Salary int
ManagerID int
}
Field order issig nificant totyp e identity.Had weals o combined the declarat ionofthe Posi-
tion eld(also a str ing), orint erc hange d Name and Address,wewou ldbedefiningadif ferent
st ruc t type.Typic ally weonlycom bine the declarat ions ofrel ate d elds.
Thenameofastr uct eldisexp orted ifitbeg inswit h acapit alletter; thisisGosmain access
cont rol mechanism. A st ruc t type may cont ain a mixtureofexp orted and unexp orted elds.
St ruc t typestendtobeverb osebecause the y of ten invo l ve alinefor each eld. Alt hough we
couldwrite out the whole typ e each timeitisneeded,the rep etit ion wou ldget tires ome.
In ste ad, str uct typ es usuallyapp ear wit hin the declarat ionofanamed typ e li ke Employee.
Anamed str uct typ e S cantdeclare a eldofthe sametyp e S:anaggregatevalue cannot con-
tain its elf.(An analogou s rest ric tionapp lies toarrays.) But S maydeclare a eldofthe
pointertyp e *S,whichlets uscre ate rec ursivedat a st ruc tures like lin ked lists andtre es. Thisis
illustrated inthe codebelow,whichusesabinar y tree toimp lement an ins ertionsor t:
gopl.io/ch4/treesort
type tree struct {
value int
left, right *tree
}
// Sort sorts values in place.
func Sort(values []int) {
var root *tree
for _, v := range values {
root = add(root, v)
}
appendValues(values[:0], root)
}
// appendValues appends the elements of t to values in order
// and returns the resulting slice.
func appendValues(values []int, t *tree) []int {
if t != nil {
values = appendValues(values, t.left)
values = append(values, t.value)
values = appendValues(values, t.right)
}
return values
}
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
102 CHAPTER 4. COMPOSITE TYPES
func add(t *tree, value int) *tree {
if t == nil {
// Equivalent to return &tree{value: value}.
t=new(tree)
t.value = value
return t
}
if value < t.value {
t.left = add(t.left, value)
}else {
t.right = add(t.right, value)
}
return t
}
Thezerovalue for a str uct iscomposedofthe zerovalues ofeachofits elds.Itisusu ally
desirable thatthe zerovalue beanatural orsensibledefau lt. For example, in bytes.Buffer,
theinitial value ofthe str uct isaready-to-use emp tybuf fer,and the zerovalue of sync.Mutex,
whichwell see inChapt er9,isaready-to-use unlocke d mu tex. Som etimesthissensibleinitial
behavior happens for fre e,but som etimesthe typ e desig ner has towor k at it.
Thestr uct typ e with nofields is cal le d the emptystr uct,writt en struct{}.Ithas size zeroand
carries noinfor mat ionbut may beusefulnon etheless. Som e Go programmersuse itins tead
of bool as the value typ e of a map thatrepresentsaset, toemp hasize thatonlythe keysare sig-
nificant,but the space sav ingismarginaland the syntaxmorecumbers ome,sowegeneral ly
avoidit.
seen := make(map[string]struct{}) // set of strings
// ...
if _, ok := seen[s]; !ok {
seen[s] = struct{}{}
// ...first time seeing s...
}
4.4.1. Struct Literals
Avalue ofastr uct typ e canbewritt enusingast ruc t litera l that specifies values for itsfields.
type Point struct{ X, Y int }
p:=Point{1, 2}
Thereare two for msofstr uct lit eral.The rs t form,shown above , re quires thatavalue be
sp ecified for ever y eld, inthe rig htorder.Itburdensthe writer(andreader) wit h rememb er-
ingexac tly whatthe elds are , anditmakes the codefrag i le shouldthe set of elds later grow
or bereordered.Accordingly, thisfor m tend s to beusedonlywit hin the packagethatdefines
thestr uct typ e,orwit h smal ler st ruc t typesfor whichthere is an obv iou s eldorder ingcon-
vent ion,like image.Point{x, y} or color.RGBA{red, green, blue, alpha}.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 4.4. STRUCTS103
Mo reoften, the secon d form isused, inwhichastr uct value isinitialize d by listing som e or all
of the eldnames andtheir correspondingvalues, as in thisstatement fro m theLissajous
prog ram of Sec tion 1.4:
anim := gif.GIF{LoopCount: nframes}
If a eldisomitt edinthiskindoflit eral,itisset tothe zerovalue for itstyp e.Because names
areprovide d,the order of elds doesntmatter.
Thetwo for mscannot bemixed inthe samelit eral.Nor can you use the (order-b ased)firs t
form oflit eral tosne akaro und the rulethatunexp orted identifiersmay not berefer red to
from another package.
package p
type T struct{ a, b int } // a and b are not exported
package q
import "p"
var _ = p.T{a: 1, b: 2} // compile error: can'treference a, b
var _ = p.T{1, 2} // compile error: can'treference a, b
Although the lastlineabove doesntmention the unexp orted eldidentifiers, itsreallyusing
them implicitly, soitsnot allow ed.
St ruc t values can bepassedasarguments tofunctions and retur ned fro m them. For ins tance,
this functionscales a Point by a specified fac tor :
func Scale(p Point, factor int) Point {
return Point{p.X * factor, p.Y * factor}
}
fmt.Println(Scale(Point{1, 2}, 5)) // "{5 10}"
Fo r efficiency,largerstr uct typ es areusu allypassedtoorretur ned fro m func tions indirec tly
usingapoint er,
func Bonus(e *Employee, percent int) int {
return e.Salary * percent / 100
}
andthisisrequired ifthe functionmustmodif y itsargument, since in a cal l-by-value langu age
li keGo, the cal le d func tionreceivesonlyacopyofanargument, not a reference tothe original
argument.
func AwardAnnualRaise(e *Employee) {
e.Salary = e.Salary * 105 / 100
}
Becaus e st ruc ts aresocommonlydealt wit h thro ugh point ers,itspossibletouse this
shorthand notation tocre ate and initialize a struct var iable andobt ain its address:
pp := &Point{1, 2}
It isexac tly equivalentto
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
104 CHAPTER 4. COMPOSITE TYPES
pp := new(Point)
*pp = Point{1, 2}
but &Point{1, 2} canbeuseddirec tly wit hin an expression, such as a functioncal l.
4.4.2. Comparing Structs
If all the elds ofa str uct are comparable,the str uct its elf iscomparable,sotwo expressions of
that typ e maybecompare d using == or !=.The == op erat ioncomparesthe cor responding
elds ofthe two str ucts in order,sothe two print edexpressions below are equivalent:
type Point struct{ X, Y int }
p:=Point{1, 2}
q:=Point{2, 1}
fmt.Println(p.X == q.X && p.Y == q.Y) // "false"
fmt.Println(p == q) // "false"
Comp arable str uct typ es, li keother comp arable typ es, maybeused as the key typ e of a map.
type address struct {
hostname string
port int
}
hits := make(map[address]int)
hits[address{"golang.org", 443}]++
4.4.3. Struct Embedding and Anonymous Fields
In thissec tion,well see how Gosunu sual st ruc t embedding me chanism lets ususe one named
st ruc t type asan an ony mou s el d of another str uct typ e,providingaconvenientsyntactic
shortcutsothatasimpledot expressionlike x.f canstand for a chain of elds like x.d.e.f.
Consider a 2-D drawingprogram thatprovides a librar y of shapes, suchasrec tangles, ellipses,
st ars,and whe els. Hereare two ofthe typ es it mig htdefine:
type Circle struct {
X, Y, Radius int
}
type Wheel struct {
X, Y, Radius, Spokes int
}
A Circle hasfields for the X and Y co ordinates ofits center, and a Radius.AWheel hasall the
fe aturesofaCircle,plu s Spokes,the numberofins cribedradi al sp okes. Letscre ate a whe el:
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 4.4. STRUCTS105
var w Wheel
w.X = 8
w.Y = 8
w.Radius = 5
w.Spokes = 20
As the set ofshapesgrows, were bound tonot ice similarities andrep etit ion among them, soit
maybeconvenienttofac tor out their common par ts:
type Point struct {
X, Y int
}
type Circle struct {
Center Point
Radius int
}
type Wheel struct {
Circle Circle
Spokes int
}
Theapp lic ationmay becle arer for it,but thischange makes accessingthe elds ofaWheel
more verb ose:
var w Wheel
w.Circle.Center.X = 8
w.Circle.Center.Y = 8
w.Circle.Radius = 5
w.Spokes = 20
Go lets usdeclare a eldwit h atyp e butnoname; such elds are cal le d an ony mou s el ds.The
type ofthe eldmustbeanamed typ e or a point ertoanamed typ e.Below, Circle and Wheel
have one anony mou s eldeach. Wesay thataPoint is embedded within Circle,and a
Circle is emb edde d within Wheel.
type Circle struct {
Point
Radius int
}
type Wheel struct {
Circle
Spokes int
}
Than ks to emb edding, wecan refer tothe names at the leavesofthe imp licittre e without
giving the int erveningnames:
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
106 CHAPTER 4. COMPOSITE TYPES
var w Wheel
w.X = 8 // equivalent to w.Circle.Point.X = 8
w.Y = 8 // equivalent to w.Circle.Point.Y = 8
w.Radius = 5 // equivalent to w.Circle.Radius = 5
w.Spokes = 20
Theexplicitfor msshown inthe commentsabove are still valid,how ever, showing that
‘‘anonymou s eld’’ is som ethingofamisnomer. The elds Circle and Point do have
namesthat ofthe named typ ebutthose names areopt ion alindot expressions.Wemay
omit any orall ofthe anony mou s elds whenselec ting their subelds.
Unfortunately, theresnocor respondingshorthand for the str uct lit eral synt ax, soneither of
thes e wi l l comp ile:
w=Wheel{8, 8, 5, 20} // compile error: unknown fields
w=Wheel{X: 8, Y: 8, Radius: 5, Spokes: 20} // compile error: unknown fields
Thestr uct lit eral mustfol low the shap e of the typ e de clarat ion, sowemustuse one ofthe two
formsbelow,whichare equivalenttoeachother :
gopl.io/ch4/embed
w=Wheel{Circle{Point{8, 8}, 5}, 20}
w=Wheel{
Circle: Circle{
Point: Point{X: 8, Y: 8},
Radius: 5,
},
Spokes: 20, // NOTE: trailing comma necessary here (and at Radius)
}
fmt.Printf("%#v\n", w)
// Output:
// Wheel{Circle:Circle{Point:Point{X:8, Y:8}, Radius:5}, Spokes:20}
w.X = 42
fmt.Printf("%#v\n", w)
// Output:
// Wheel{Circle:Circle{Point:Point{X:42, Y:8}, Radius:5}, Spokes:20}
No tice how the # adverb causes Printfs %v verb todispl ayvalues in a for m simi lar toGosyn-
tax. For str uct values, thisfor m includes the nameofeach eld.
Becaus e ‘‘anonymou s’’ elds dohaveimp licitnames, you canthavetwo anony mou s elds of
thesametyp e since their names wou ldconflic t. Andbecause the nameofthe eldisimp lic-
it lydeter mined byits typ e,sotoo isthe visibilit y of the eld. Inthe examples abov e,the Point
and Circle anonymou s elds are exp orted.Had the y been unexp orted (point and circle),
we cou ldstill use the shorthand for m
w.X = 8 // equivalent to w.circle.point.X = 8
butthe explicitlon g form shown inthe comment wou ldbeforbidden outside the declaring
packagebecause circle and point wouldbeinaccessible.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 4.5. JSON 107
Wh atweve seensofar ofstr uct emb eddingisjustaspr ink lingofsyntactic sugar onthe dot
notation usedtoselec t st ruc t elds.Later,well see thatanony mou s elds need not bestr uct
types; anynamed typ e or point ertoanamed typ e wi l l do.But why wou ldyou wanttoemb ed
atyp e that has nosubelds?
Theanswerhas to do wit h methods. Theshorthand notation usedfor selec ting the elds ofan
embedde d type wor ksfor selec ting its met hodsaswel l.Ineffec t,the out erstr uct typ e gains
notjustthe elds ofthe emb edde d type but itsmet hodstoo.Thismechanism isthe main way
that comp lex obj e ctbeh avior s arecomposedfro m simpler ones. Comp ositi on is central to
objec t-oriente d prog ramming in Go, and well explore itfur ther in Sec tion 6.3.
4.5. JSON
JavaS crip t Objec t No tat ion(JSON) isastand ard not ation for sendingand receiving str uctured
infor mat ion. JSONisnot the onlysuchnot ation.XML (§7.14), ASN.1, andGooglesPro tocol
Buffersser vesimi lar pur pos es andeachhas its niche,but because ofits simplicity,readabi lit y,
andunivers alsup por t,JSONisthe most widely used.
Go has excellentsup por t forencodingand decodingthese for mats, provide d by the stand ard
librar y packages encoding/json, encoding/xml, encoding/asn1,and soon, andthese pack-
ages allhavesimi lar APIs.Thissec tion gives a brief overvie w of the most imp ortantpar ts of
the encoding/json package.
JSONisanencodingofJavaS crip t valuesst rings, numbers,boole ans, arrays, andobj e ctsas
Unico de text. Itsanefficientyet readablerepresent ation for the basic dat a typesofChapt er3
andthe composite typ es of thischapt erar rays, slices, str ucts, andmaps.
Thebasic JSONtyp es arenumbers (in decimal orscienticnot ation), boole ans(true or
false), andstr ings, whichare sequences ofUnico de co de points enclos edindou ble quotes,
with backslash escap es usingasimi lar not ation toGo, thoug h JSONs \Uhhhh numericescap es
denot e UTF-16 codes, not runes.
Thes e basic typ es maybecom bine d re cursive lyusingJSONarrays andobj e cts. A JSONarray
is anordered sequence ofvalues, writt enasa comma-s eparated listenclos edinsqu are brack-
ets; JSONarrays areusedtoencodeGoarrays andslices. A JSONobj e ctisamapping fro m
st rings tovalues, writt enasasequence of name:value pairssep arated bycommasand sur-
ro unded bybraces; JSONobj e cts areusedtoencodeGomaps (wit h st ringkeys) andstr ucts.
Fo r example:
boolean true
number -273.15
string "She said \"Hello, BF\""
array ["gold", "silver", "bronze"]
object {"year": 1980,
"event": "archery",
"medals": ["gold", "silver", "bronze"]}
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
108 CHAPTER 4. COMPOSITE TYPES
Consider an applic ationthatgat hersmov ierevie ws andoffersrecommend ation s.Its Movie
data typ e andatypic al list ofvalues aredeclare d below. (Thestr ing lit eralsafter the Year and
Color elddeclarat ions are el d tags;well explain them in a mom ent.)
gopl.io/ch4/movie
type Movie struct {
Title string
Year int `json:"released"`
Color bool `json:"color,omitempty"`
Actors []string
}
var movies = []Movie{
{Title: "Casablanca", Year: 1942, Color: false,
Actors: []string{"Humphrey Bogart", "Ingrid Bergman"}},
{Title: "Cool Hand Luke", Year: 1967, Color: true,
Actors: []string{"Paul Newman"}},
{Title: "Bullitt", Year: 1968, Color: true,
Actors: []string{"Steve McQueen", "Jacqueline Bisset"}},
// ...
}
Data str uctures like thisare anexcel lent tfor JSON, anditseasy toconvert inbot h
direc tion s.ConvertingaGodat a st ruc turelike movies to JSONiscal le d marshaling.Mar-
sh alingisdon e by json.Marshal:
data, err := json.Marshal(movies)
if err != nil {
log.Fatalf("JSON marshaling failed: %s", err)
}
fmt.Printf("%s\n", data)
Marshal produces a byteslice cont ainingaver y long str ing wit h no ext raneous white space;
weve folde d thelines soit fits:
[{"Title":"Casablanca","released":1942,"Actors":["Humphrey Bogart","Ingr
id Bergman"]},{"Title":"Cool Hand Luke","released":1967,"color":true,"Ac
tors":["Paul Newman"]},{"Title":"Bullitt","released":1968,"color":true,"
Actors":["Steve McQueen","Jacqueline Bisset"]}]
Thiscompact represent ation containsall the infor mat ionbut itshardtoread. For human
cons ump tion, a variant cal le d json.MarshalIndent produces neatlyindente d output.Two
addition alarguments define a prexfor eachlineofout put and a str ing for eachlevel of
indentation:
data, err := json.MarshalIndent(movies, "", " ")
if err != nil {
log.Fatalf("JSON marshaling failed: %s", err)
}
fmt.Printf("%s\n", data)
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 4.5. JSON 109
Thecodeabove prints
[
{
"Title": "Casablanca",
"released": 1942,
"Actors": [
"Humphrey Bogart",
"Ingrid Bergman"
]
},
{
"Title": "Cool Hand Luke",
"released": 1967,
"color": true,
"Actors": [
"Paul Newman"
]
},
{
"Title": "Bullitt",
"released": 1968,
"color": true,
"Actors": [
"Steve McQueen",
"Jacqueline Bisset"
]
}
]
Mars halingusesthe Gostr uct eldnames as the eldnames for the JSONobj e cts (through
reec tion,aswell see inSec tion 12.6). Only exp orted elds are marsh ale d,whichiswhy we
ch ose capit alize d namesfor all the Gofieldnames.
Yo u mayhavenot ice d that the nameofthe Year eldchange d to released in the out put,and
Color ch ange d to color.Thatsbecause ofthe el d tags.Aeldtag isastr ing ofmet adat a
asso ciate d at comp ile timewit h thefieldofastr uct:
Year int `json:"released"`
Color bool `json:"color,omitempty"`
Afieldtag may beany lit eral str ing , butitisconvent ion allyint erprete d as a space-s eparated
list of key:"value" pairs; since the y cont ain double quotation marks, eldtags areusu ally
wr itt enwit h raw str ing lit erals. The json ke y cont rol s thebeh avior of the encoding/json
package, and other encoding/... packages fol low thisconvent ion.The rs t part ofthe json
eldtag specifies an alternat ive JSONnamefor the Gofield. Field tags areoften usedto
sp ecif y an idiomaticJSONnamelike total_count foraGofieldnamed TotalCount.The tag
for Color has an addition alopt ion, omitempty,whichindic ates thatnoJSONout put should
be pro duce d if the eldhas the zerovalue for itstyp e (false,here) orisother wis e empt y.
Sure enoug h, theJSONout put for Casabl anca,ablack-and-w hite mov ie, has no color eld.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
110 CHAPTER 4. COMPOSITE TYPES
Theinv ers e op erat iontomarsh aling, decodingJSONand popu lat ingaGodat a st ruc ture, is
called unmarshaling,and itisdon e by json.Unmarshal.The codebelow unmarsh als the
JSONmov iedat a into a slice ofstr uctswhose onlyfieldis Title.BydefiningsuitableGodat a
st ruc tures in thisway,wecan selec t whichpar ts of the JSONinp uttodecodeand whichtodis-
card . Wh en Unmarshal returns, ithas lled inthe slice wit h the Title infor mat ion; other
names in the JSONare ignored.
var titles []struct{ Title string }
if err := json.Unmarshal(data, &titles); err != nil {
log.Fatalf("JSON unmarshaling failed: %s", err)
}
fmt.Println(titles) // "[{Casablanca} {Cool Hand Luke} {Bullitt}]"
Many web ser vices provide a JSONint erfacemake a requestwit h HT TP andbackcom esthe
desired infor mat ioninJSONfor mat. Toillustrate, letsquer y theGitHu b issuetrackerusing
itsweb-s ervice interface.First well define the necessary typ es andcon stants:
gopl.io/ch4/github
// Package github provides a Go API for the GitHub issue tracker.
// See https://developer.github.com/v3/search/#search-issues.
package github
import "time"
const IssuesURL = "https://api.github.com/search/issues"
type IssuesSearchResult struct {
TotalCount int `json:"total_count"`
Items []*Issue
}
type Issue struct {
Number int
HTMLURL string `json:"html_url"`
Title string
State string
User *User
CreatedAt time.Time `json:"created_at"`
Body string // in Markdown format
}
type User struct {
Login string
HTMLURL string `json:"html_url"`
}
As before, the names ofall the str uct elds mustbecapit alize d even if their JSONnames are
not. How ever, the matchingpro cessthatass oci ates JSONnames wit h Go str uct names dur ing
unmarsh alingiscas e-insensit ive , so itsonlynecessary touse a eldtag whentheresanunder-
scoreinthe JSONnamebut not inthe Goname. Again, weare beingselec tive about which
elds todecode; the GitHu b search respons e cont ainscon siderably moreinfor mat ionthanwe
showhere.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 4.5. JSON 111
The SearchIssues func tionmakes an HTTP requestand decodes the resultasJSON. Since
thequer y termspresent edbyausercou ldcontain charac terslike ? and & that havespeci al
me aning in a URL, weuse url.QueryEscape to ens ure thatthe y aretaken literal ly.
gopl.io/ch4/github
package github
import (
"encoding/json"
"fmt"
"net/http"
"net/url"
"strings"
)
// SearchIssues queries the GitHub issue tracker.
func SearchIssues(terms []string) (*IssuesSearchResult, error) {
q:=url.QueryEscape(strings.Join(terms, " "))
resp, err := http.Get(IssuesURL + "?q=" + q)
if err != nil {
return nil, err
}
// We must close resp.Body on all execution paths.
// (Chapter 5 presents 'defer',which makes this simpler.)
if resp.StatusCode != http.StatusOK {
resp.Body.Close()
return nil, fmt.Errorf("search query failed: %s", resp.Status)
}
var result IssuesSearchResult
if err := json.NewDecoder(resp.Body).Decode(&result); err != nil {
resp.Body.Close()
return nil, err
}
resp.Body.Close()
return &result, nil
}
Theearlier examples used json.Unmarshal to decodethe ent ire contentsofabyteslice as a
singleJSONent ity.For var iety, thisexampleusesthe st reami n g de coder, json.Decoder,
whichallowsseveral JSONent ities tobedecoded insequence fro m thesamestream, although
we dontneed thatfeature here. Asyou mig htexp ect,there isacor respondingstreaming
enco der called json.Encoder.
Thecal l to Decode popu lates the var iable result.There are var ious ways wecan for mat its
value nicely.The simplest, demon strated bythe issues commandbelow,isasatext table
with xe d-w idth columns, but inthe next sec tion well see a moresop histicate d approach
basedontempl ates.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
112 CHAPTER 4. COMPOSITE TYPES
gopl.io/ch4/issues
// Issues prints a table of GitHub issues matching the search terms.
package main
import (
"fmt"
"log"
"os"
"gopl.io/ch4/github"
)
func main() {
result, err := github.SearchIssues(os.Args[1:])
if err != nil {
log.Fatal(err)
}
fmt.Printf("%d issues:\n", result.TotalCount)
for _, item := range result.Items {
fmt.Printf("#%-5d %9.9s %.55s\n",
item.Number, item.User.Login, item.Title)
}
}
Thecommand-linearguments specif y thesearc h terms. Thecommand below quer ies theGo
proj e ctsissuetrackerfor the listofopenbugs rel ate d to JSONdecoding:
$gobuild gopl.io/ch4/issues
$./issues repo:golang/go is:open json decoder
13 issues:
#5680 eaigner encoding/json: set key converter on en/decoder
#6050 gopherbot encoding/json: provide tokenizer
#8658 gopherbot encoding/json: use bufio
#8462 kortschak encoding/json: UnmarshalText confuses json.Unmarshal
#5901 rsc encoding/json: allow override type marshaling
#9812 klauspost encoding/json: string tag not symmetric
#7872 extempora encoding/json: Encoder internally buffers full output
#9650 cespare encoding/json: Decoding gives errPhase when unmarshalin
#6716 gopherbot encoding/json: include field name in unmarshal error me
#6901 lukescott encoding/json, encoding/xml: option to treat unknown fi
#6384 joeshaw encoding/json: encode precise floating point integers u
#6647 btracey x/tools/cmd/godoc: display type kind of each named type
#4237 gjemiller encoding/base64: URLEncoding padding is optional
TheGitHu b we b-s ervice interface at https://developer.github.com/v3/ hasmanymore
fe aturesthanwehavespace for here.
Exercis e 4.10: Mo dif y issues to rep ort the results in agecategor ies, say lessthanamonth old,
lessthanayearold,and more thanayearold.
Exercis e 4.11: Buildatoolthatlets users cre ate, read, update, and deleteGitHu b issues fro m
thecommand line, inv oking their preferred text editorwhensubst ant ial text inputisrequired.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 4.6. TEXT AND HTML TEMPLATES 113
Exercis e 4.12: Thepopu lar web comic xkcd hasaJSONint erface.For example, a requestto
https://xkcd.com/571/info.0.json produces a detai le d des crip tionofcomic 571, one of
many favor ites. Dow nlo ad each URL (once!) andbui ld an offlineindex. Write a tool xkcd
that, usingthisindex, printsthe URL andtranscript ofeachcomic thatmatch esasearc h term
prov ide d on the command line.
Exercis e 4.13: TheJSON-b ased web ser vice ofthe OpenMov ieDat abas e lets you searc h
https://omdbapi.com/ foramov iebynameand dow nlo ad itsposterimage . Wr ite a tool
poster that down loads the posterimage for the movienamed onthe command line.
4.6. Textand HTML Templates
Thepre vious exampledoesonlythe simplestpossiblefor matting , forwhich Printf is ent ire ly
ade quate. But som etimesfor matting mustbemoreelaborate, and itsdesirable tosep aratethe
format fro m thecodemorecompletely. Thiscan bedon e with the text/template and
html/template packages, whichprovide a mechanism for subst ituting the values ofvar iables
into a text orHTMLtempl ate.
Atempl ate isastr ing orfile cont ainingone ormorepor tions enclos edindou ble braces,
{{...}},cal le d ac tions.Mostofthe str ing isprint edlit eral ly, but the actions trigger other
behavior s.Eachactioncontainsanexpressioninthe templ ate langu age, a simplebut pow erful
notation for printing values, selec ting str uct elds,cal lingfunctions and methods, expressing
cont rol ow suchas if-else st atementsand range lo ops, andins tantiat ingother templ ates.
Asimpletempl ate str ing isshown below :
gopl.io/ch4/issuesreport
const templ = `{{.TotalCount}} issues:
{{range .Items}}----------------------------------------
Number: {{.Number}}
User: {{.User.Login}}
Title: {{.Title |printf "%.64s"}}
Age: {{.CreatedAt |daysAgo}} days
{{end}}`
Thistempl ate rs t pr intsthe numberofmatchingissues, thenprintsthe number, user, tit le,
andage indays ofeachone.Wit hin an action,there isanot ion of the cur rentvalue,refer red
to as ‘‘dot’’ andwritt enas ‘‘.’’,aper iod. The dot initial lyreferstothe templ atesparameter,
whichwill beagithub.IssuesSearchResult in thisexample. The {{.TotalCount}} ac tion
exp ands tothe value ofthe TotalCount eld, print edinthe usu alway.The
{{range .Items}} and {{end}} ac tions cre ate a loop, sothe text bet weenthemisexp ande d
mu ltipletimes, wit h dotbound tosuccessive elements of Items.
Wi thin an action,the | notation makes the resultofone operat ionthe argumentofanother,
analogou s to a Unix shel l pip eline. Inthe cas e of Title,the secon d op erat ionisthe printf
func tion, whichisabui lt-in synony m for fmt.Sprintf in alltempl ates. For Age,the secon d
op erat ionisthe fol low ing function, daysAgo,whichconvertsthe CreatedAt eldint o an
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
114 CHAPTER 4. COMPOSITE TYPES
el aps edtime, using time.Since:
func daysAgo(t time.Time) int {
return int(time.Since(t).Hours() / 24)
}
No tice thatthe typ e of CreatedAt is time.Time,not string.Inthe sameway thatatyp e may
cont rol its str ing for matting (§2.5) bydefiningcer tain met hods, a typ e mayals o define meth-
odstocontrol its JSONmarsh alingand unmarsh alingbeh avior.The JSON-marsh ale d value
of a time.Time is a str ing inastand ard for mat.
Producingout put wit h atempl ate isatwo-steppro cess. First wemustparse the templ ate int o
asuitableint ernal represent ation,and thenexe cut e it on specificinp uts. Parsingneed bedon e
on lyonce. The codebelow cre atesand parsesthe templ ate templ define d ab ove . No tethe
ch ainingofmet hod cal ls: template.New createsand retur nsatempl ate; Funcs adds daysAgo
to the set offunctions accessiblewit hin thistempl ate, thenretur nsthattempl ate;finally, Parse
is cal le d on the result.
report, err := template.New("report").
Funcs(template.FuncMap{"daysAgo": daysAgo}).
Parse(templ)
if err != nil {
log.Fatal(err)
}
Becaus e temp lates areusu allyfixe d at comp ile time, fai luretoparse a templ ate indic ates a fatal
buginthe program. The template.Must helperfunctionmakes erro r hand lingmorecon-
venient: itaccepts a templ ate and anerror,che cks thatthe error isnil (andpanics other wis e),
andthenretur nsthe templ ate. Well com e back to thiside a in Sec tion 5.9.
Once the templ ate has beencre ate d, aug mente d with daysAgo,parsed, and che cke d,wecan
exec ute itusingagithub.IssuesSearchResult as the dat a source and os.Stdout as the des-
tination:
var report = template.Must(template.New("issuelist").
Funcs(template.FuncMap{"daysAgo": daysAgo}).
Parse(templ))
func main() {
result, err := github.SearchIssues(os.Args[1:])
if err != nil {
log.Fatal(err)
}
if err := report.Execute(os.Stdout, result); err != nil {
log.Fatal(err)
}
}
Theprogram printsaplain text rep ort likethis:
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 4.6. TEXT AND HTML TEMPLATES 115
$gobuild gopl.io/ch4/issuesreport
$./issuesreport repo:golang/go is:open json decoder
13 issues:
----------------------------------------
Number: 5680
User: eaigner
Title: encoding/json: set key converter on en/decoder
Age: 750 days
----------------------------------------
Number: 6050
User: gopherbot
Title: encoding/json: provide tokenizer
Age: 695 days
----------------------------------------
...
No w letstur n to the html/template package. Itusesthe sameAPI and expressionlangu age
as text/template butaddsfeaturesfor aut omat ic andcontext-appropriateescapingofstr ings
appear ingwit hin HTML, JavaS crip t,CSS, orURLs. Thes e fe aturescan helpavoid a perenni al
security pro blem ofHTMLgenerat ion, an injectionattack, in which an adversary craf ts a
st ringvalue like the tit leofanissuetoinclude malicious codethat, whenimp rop erlyescap ed
by a templ ate, gives themcontrol overthe page.
Thetempl ate below printsthe listofissues as an HTML table.Not e thedif ferentimp ort:
gopl.io/ch4/issueshtml
import "html/template"
var issueList = template.Must(template.New("issuelist").Parse(`
<h1>{{.TotalCount}} issues</h1>
<table>
<tr style='text-align: left'>
<th>#</th>
<th>State</th>
<th>User</th>
<th>Title</th>
</tr>
{{range .Items}}
<tr>
<td><a href='{{.HTMLURL}}'>{{.Number}}</td>
<td>{{.State}}</td>
<td><a href='{{.User.HTMLURL}}'>{{.User.Login}}</a></td>
<td><a href='{{.HTMLURL}}'>{{.Title}}</a></td>
</tr>
{{end}}
</table>
`))
Thecommand below exe cut esthe newtempl ate onthe results ofaslig htlydif ferentquer y:
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
116 CHAPTER 4. COMPOSITE TYPES
$gobuild gopl.io/ch4/issueshtml
$./issueshtml repo:golang/go commenter:gopherbot json encoder >issues.html
Figure4.4 shows the app earance ofthe table in a web brows er.The lin ks connec t to the
appropriateweb pages at GitHu b.
Figure 4.4. An HTMLtable ofGopro jec t issues rel ating toJSONencoding.
No neofthe issues in Figure4.4 pos e achallenge for HTML, but wecan see the effec t more
clearlywit h issues whose tit les cont ain HTML met acharac terslike & and <.Weve selec ted two
such issues for thisexample:
$./issueshtml repo:golang/go 3133 10535 >issues2.html
Figure4.5 shows the resultofthisquer y.Not ice thatthe html/template packageaut omat i-
callyHTML-es cap edthe tit les sothatthe y appear literal ly. Had weusedthe text/template
packagebymistake , thefour-ch arac ter str ing "&lt;" wouldhavebeenrendered asaless-t han
ch arac ter '<',and the str ing "<link>" wouldhavebecom e a link element, chang ing the
st ruc tureofthe HTMLdocumentand perhaps comp romisingits sec urity.
We can sup pressthisaut o-es capingbeh avior for elds thatcontain trusted HTMLdat a by
usingthe named str ing typ e template.HTML insteadof string.Simi lar named typ es exist for
tr usted JavaS crip t,CSS, andURLs. Theprogram below demon strates the princip lebyusing
twofields wit h thesamevalue but dif ferenttyp es: A is a str ing and B is a template.HTML.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 4.6. TEXT AND HTML TEMPLATES 117
Figure 4.5. HTML met acharac ters in issuetit les arecor rec tly displ aye d.
gopl.io/ch4/autoescape
func main() {
const templ = `<p>A: {{.A}}</p><p>B: {{.B}}</p>`
t:=template.Must(template.New("escape").Parse(templ))
var data struct {
Astring // untrusted plain text
Btemplate.HTML // trusted HTML
}
data.A = "<b>Hello!</b>"
data.B = "<b>Hello!</b>"
if err := t.Execute(os.Stdout, data); err != nil {
log.Fatal(err)
}
}
Figure4.6 shows the templ atesout put asitapp earsinabrows er.Wecan see that A was
su bjec t to escapingbut B was not.
Figure 4.6. St ringvalues areHTML-es cap edbut template.HTML values arenot.
We havespace heretoshowonlythe most basic featuresofthe templ ate system. Asalways, for
more infor mat ion, cons ult the packagedocumentation:
$godoc text/template
$godoc html/template
Exercis e 4.14: Create a web ser ver thatquer ies GitHub onceand thenallowsnav igationofthe
list ofbug rep orts, milestones, andusers.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
This page intentionally left blank
From the Library of YIGUANG HU
ptg16105617
5
Functions
Afunctionlets uswrap upasequence ofstatementsasaunitthatcan becal le d from els e where
in a program, perhaps multipletimes. Functions makeitpossibletobre akabig job int o
smal ler pieces thatmig htwel l be writt enbydif ferentpeoplesep arated bybot h time and space.
Afunctionhides its implementation det ails fro m itsusers.For all ofthese reasons,functions
areacriticalpar t of any programminglangu age.
Weve seenmanyfunctions already.Now letstaketimefor a morethoro ugh dis cussion. The
runningexampleofthischapt erisaweb craw ler,thatis, the component ofaweb searc h
engine responsible for fetchingweb pages, discov ering the lin ks within them, fetchingthe
pagesidentied bythose lin ks, andsoon. A we b craw ler givesusamp leopp ortunity to
explore recursion, anonymou s func tions,error handling, and asp ectsoffunctions thatare
unique toGo.
5.1. Function Declarations
Afunctiondeclarat ionhas a name, a listofparameters, an opt ion allistofresults, andabody:
func
name
(
parameter-list
)(
result-list
){
body
}
Theparameter list specifies the names andtyp es of the functions para meters,whichare the
lo cal variables whose values or ar gum ents aresup plie d by the cal ler.The resultlistspecifies
thetyp es of the values thatthe functionretur ns. Ifthe functionretur nsone unnamed result
or noresults at all, parenthes es areopt ion aland usu allyomitt ed. Leaving off the resultlist
entire lydeclaresafunctionthatdoesnot retur n anyvalue andiscal le d on lyfor itseffec ts. In
the hypot func tion,
119
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
120 CHAPTER 5. FUNCTIONS
func hypot(x, y float64) float64 {
return math.Sqrt(x*x + y*y)
}
fmt.Println(hypot(3, 4)) // "5"
x and y areparametersinthe declarat ion, 3 and 4 arearguments ofthe cal l,and the function
returnsafloat64 value.
Like parameters, results may benamed.Inthatcas e,eachnamedeclaresalocal variableini-
tialize d to the zerovalue for itstyp e.
Afunctionthathas a resultlistmustend wit h a return st atement unlessexe cut ion cle arly
cannot reach the end ofthe function, perhaps because the functionend s with a cal l to panic
or aninfinite for lo opwit h no break.
As wesaw wit h hypot,asequence ofparametersorresults ofthe sametyp e canbefac tored so
that the typ e itself iswritt enonlyonce. These two declarat ions are equivalent:
func f(i, j, k int, s, t string) {/*... */ }
func f(i int, j int, k int, s string, t string) { /* ... */ }
Here are fourways todeclare a functionwit h twoparametersand one result, alloftyp e int.
Theblank identifier can beusedtoemp hasize thataparameter isunu sed.
func add(x int, y int) int {return x + y }
func sub(x, y int) (z int) {z=x-y;return }
func first(x int, _ int) int { return x }
func zero(int, int) int {return 0 }
fmt.Printf("%T\n", add) // "func(int, int) int"
fmt.Printf("%T\n", sub) // "func(int, int) int"
fmt.Printf("%T\n", first) // "func(int, int) int"
fmt.Printf("%T\n", zero) // "func(int, int) int"
Thetyp e of a functionissom etimescal le d its sig nat ure.Two functions havethe sametyp e or
sig nature if the y have the samesequence ofparameter typ es andthe samesequence ofresult
types. Thenames ofparametersand results dontaffec t thetyp e,nor doeswhether ornot the y
were declare d usingthe fac tored for m.
Ev ery functioncal l mu stprovide an argumentfor eachparameter,inthe order in whichthe
parametersweredeclare d.Gohas nocon ceptofdefau ltparameter values, nor any way to
sp ecif y arguments byname, sothe names ofparametersand results dontmattertothe cal ler
except as documentation.
Parametersare local variables wit hin the bodyofthe function, wit h their initial values set to
thearguments sup plie d by the cal ler.Functionparametersand named results arevar iables in
thesamelexic al block as the functionsout ermostlocal variables.
Arguments arepassed by value,sothe functionreceivesacopyofeachargument; modifica-
tion s to the copy do not affec t thecal ler.How ever, ifthe argumentcontainssom e kind ofref-
erence,likea point er, slice,map,function, orchannel,thenthe cal ler maybeaffec ted byany
mo dification s thefunctionmakes tovar iables in direc tly referred tobythe argument.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 5.2. RECURSION 121
Yo u mayocc asionallyencount erafunctiondeclarat ionwit houtabody, indic atingthatthe
func tionisimp lemente d in a langu ageother thanGo. Suchadeclarat iondefinesthe function
sig nature.
package math
func Sin(x float64) float64 // implemented in assembly language
5.2. Recursion
Func tions may be re cur siv e,thatis, the y maycal l themselves, either direc tly orindirec tly.
Re cursionisapow erfultechnique for manypro blems, andofcours e itsess ent ial for process-
ingrec ursivedat a st ruc tures. InSec tion 4.4, weusedrec ursionoveratree toimp lementa
simpleins ertionsor t.Inthissec tion,well use itagain for processingHTMLdocuments.
Theexampleprogram below usesanon-stand ard package, golang.org/x/net/html,which
prov ides an HTML parser. The golang.org/x/... repositories holdpackages designe d and
maintained bythe Goteamfor app lic ations suchasnet wor king, int ernat ionalize d text
processing, mobileplatfor ms, imagemanipu lat ion, cryptography,and develop ertools.These
packages arenot inthe stand ard librar y becaus e theyre still under deve lopment orbecause
theyre rarelyneeded bythe maj ority ofGoprogrammers.
Thepar ts of the golang.org/x/net/html APIthatwell need are shown below.The function
html.Parse re ads a sequence ofbytes, parsesthem, andretur nsthe rootofthe HTMLdoc-
umenttre e,whichisan html.Node.HTMLhas several kinds ofnodestext, comments, and
so onbuthereweare con cer ned onlywit h el ement no des ofthe for m <name key='value'>.
golang.org/x/net/html
package html
type Node struct {
Type NodeType
Data string
Attr []Attribute
FirstChild, NextSibling *Node
}
type NodeType int32
const (
ErrorNode NodeType = iota
TextNode
DocumentNode
ElementNode
CommentNode
DoctypeNode
)
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
122 CHAPTER 5. FUNCTIONS
type Attribute struct {
Key, Val string
}
func Parse(r io.Reader) (*Node, error)
The main func tionparsesthe stand ard inp utasHTML, extrac ts thelin ks usingarec ursive
visit func tion, andprintseachdis cov ere d lin k:
gopl.io/ch5/findlinks1
// Findlinks1 prints the links in an HTML document read from standard input.
package main
import (
"fmt"
"os"
"golang.org/x/net/html"
)
func main() {
doc, err := html.Parse(os.Stdin)
if err != nil {
fmt.Fprintf(os.Stderr, "findlinks1: %v\n", err)
os.Exit(1)
}
for _, link := range visit(nil, doc) {
fmt.Println(link)
}
}
The visit func tiontraversesanHTMLnodetre e,ext racts the lin k from the href attr ibute of
each an chor element <a href='...'>,app end s thelin ks to a slice ofstr ings, andretur nsthe
resu lting slice:
// visit appends to links each link found in n and returns the result.
func visit(links []string, n *html.Node) []string {
if n.Type == html.ElementNode && n.Data == "a" {
for _, a := range n.Attr {
if a.Key == "href" {
links = append(links, a.Val)
}
}
}
for c := n.FirstChild; c != nil; c = c.NextSibling {
links = visit(links, c)
}
return links
}
To des cendthe tre e fora node n, visit re cursive lycal lsits elf for eachof nschi ldren, which
areheld in the FirstChild lin ked list.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 5.2. RECURSION 123
Letsrun findlinks on the Gohom e page , piping the out put of fetch (§1.5) tothe inp utof
findlinks.Weve edite d theout put slig htlyfor brevit y.
$gobuild gopl.io/ch1/fetch
$gobuild gopl.io/ch5/findlinks1
$./fetch https://golang.org | ./findlinks1
#
/doc/
/pkg/
/help/
/blog/
http://play.golang.org/
//tour.golang.org/
https://golang.org/dl/
//blog.golang.org/
/LICENSE
/doc/tos.html
http://www.google.com/intl/en/policies/privacy/
No tice the var ietyoffor msoflin ks that appear in the page. Later well see how tores olve
them rel ative tothe bas e URL, https://golang.org,tomakeabs olut e URLs.
Thenext program usesrec ursionoverthe HTMLnodetre e to print the str uctureofthe tre e in
outline. Asitencount ers eachelement, itpushesthe elementstag ont o astack,thenprintsthe
st ack.
gopl.io/ch5/outline
func main() {
doc, err := html.Parse(os.Stdin)
if err != nil {
fmt.Fprintf(os.Stderr, "outline: %v\n", err)
os.Exit(1)
}
outline(nil, doc)
}
func outline(stack []string, n *html.Node) {
if n.Type == html.ElementNode {
stack = append(stack, n.Data) // push tag
fmt.Println(stack)
}
for c := n.FirstChild; c != nil; c = c.NextSibling {
outline(stack, c)
}
}
No teone subtlet y:alt hough outline ‘‘pu shes’’ an elementon stack,there isnocor responding
pop. When outline callsits elf rec ursively, the cal le e re ceivesacopyof stack.Alt hough the
callee may append elements tothisslice,modif yingits underly ing array andperhaps even
al locating a new array,itdoesntmodif y theinitial elements thatare visible tothe cal ler,so
when the functionretur ns, the cal lers stack is asitwas beforethe cal l.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
124 CHAPTER 5. FUNCTIONS
Heresthe out lineof https://golang.org,again edite d forbre vit y:
$gobuild gopl.io/ch5/outline
$./fetch https://golang.org | ./outline
[html]
[html head]
[html head meta]
[html head title]
[html head link]
[html body]
[html body div]
[html body div]
[html body div div]
[html body div div form]
[html body div div form div]
[html body div div form div a]
...
As you can see byexp erimenting wit h outline,mostHTMLdocuments can bepro cessed
with onlyafew level s of rec ursion, but itsnot hardtocon str uct pat holog ical web pages that
re quireext remelydeeprec ursion.
Many programminglangu ageimp lementation s us e afixe d-size functioncal l st ack;sizes fro m
64KB to2MB aretypic al.Fixed-size stacksimp ose a limitonthe depth ofrec ursion, soone
mu stbecaref ultoavoid a sta ckoverflow when traversinglarge dat a st ruc tures rec ursively;
xe d-size stacksmay evenpos e asec urity risk. Incontrast, typic al Go imp lementation s us e
var iable-size stacksthatstart small and grow as needed uptoalimitonthe order ofagigabyte.
Thislets ususe rec ursionsafelyand wit houtwor rying about overflow.
Exercis e 5.1: Change the findlinks prog ram to traverse the n.FirstChild lin ked listusing
re cursive cal lsto visit insteadofaloop.
Exercis e 5.2: Wr ite a functiontopopu lateamapping fro m elementnamesp, div, span,and
so onto the numberofelements wit h that name in an HTMLdocumenttre e.
Exercis e 5.3: Wr ite a functiontoprint the contentsofall text nodes in an HTML document
tree.Donot des cendint o <script> or <style> elements, since their cont entsare not visible
in a web brows er.
Exercis e 5.4: Extend the visit func tionsothatitext racts other kinds oflin ks from the doc-
ument, such as images, scr ipts, andsty leshe ets.
5.3. Multiple Return Values
Afunctioncan retur n more thanone result. Weve seenmanyexamples offunctions fro m
st and ard packages thatretur n twovalues, the desired computation alresultand anerror value
or boole an that indic ates whether the computation wor ked.The next exampleshows how to
wr ite one ofour own.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 5.3. MULTIPLE RETURN VALUES 125
Theprogram below isavar iat ionof findlinks that makes the HTTPrequestits elf sothatwe
no lon g erneed torun fetch.Because the HTTPand parsingoperat ions can fail, findLinks
de clarestwo results: the listofdis cov ere d lin ks andanerror.Incidentally, the HTMLparser
canusu allyrecov erfro m badinp utand con str uct a documentcontainingerror nodes, so
Parse rarelyfai ls; whenitdoes, itstypic ally due tounderly ing I/O erro rs.
gopl.io/ch5/findlinks2
func main() {
for _, url := range os.Args[1:] {
links, err := findLinks(url)
if err != nil {
fmt.Fprintf(os.Stderr, "findlinks2: %v\n", err)
continue
}
for _, link := range links {
fmt.Println(link)
}
}
}
// findLinks performs an HTTP GET request for url, parses the
// response as HTML, and extracts and returns the links.
func findLinks(url string) ([]string, error) {
resp, err := http.Get(url)
if err != nil {
return nil, err
}
if resp.StatusCode != http.StatusOK {
resp.Body.Close()
return nil, fmt.Errorf("getting %s: %s", url, resp.Status)
}
doc, err := html.Parse(resp.Body)
resp.Body.Close()
if err != nil {
return nil, fmt.Errorf("parsing %s as HTML: %v", url, err)
}
return visit(nil, doc), nil
}
Thereare fourretur n st atementsin findLinks,eachofwhichretur nsapair ofvalues. The
rs t thre e returnscause the functiontopassthe underly ing error s from the http and html
packages ontothe cal ler.Inthe rs t case,the error isretur ned unchange d;inthe secon d and
third, itisaug mente d with addition alcontext infor mat ionby fmt.Errorf (§7.8). If find-
Links is successf ul, the nalretur n st atement retur nsthe slice oflin ks, with noerror.
We mustens ure that resp.Body is clos edsothatnet wor k resources arepro perly releasedeven
in cas e of error.Gosgarb agecol lec tor rec ycles unus edmemor y,but donot assume itwill
re lease unu sed operat ingsystemres ources like openfiles andnet wor k connec tion s.The y
shouldbeclos edexplicitly.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
126 CHAPTER 5. FUNCTIONS
Theresultofcal lingamulti-value d func tionisatup leofvalues. Thecal ler of suchafunction
mu stexplicitlyassig n thevalues tovar iables if anyofthemare tobeused:
links, err := findLinks(url)
To ignoreone ofthe values, assig n it tothe blank identifier :
links, _ := findLinks(url) // errors ignored
Theresultofamulti-value d call may its elf beretur ned fro m a(mu lti-value d) callingfunction,
as in thisfunctionthatbeh aveslike findLinks butlogs its argument:
func findLinksLog(url string) ([]string, error) {
log.Printf("findLinks %s", url)
return findLinks(url)
}
Amulti-value d call may appear as the soleargumentwhencal lingafunctionofmultiple
parameters. Alt hough rarelyusedinpro duc tioncode, thisfeature issom etimesconvenient
during debug gingsince itlets usprint all the results ofacal l usingasinglestatement.The two
pr int statementsbelow havethe sameeffec t.
log.Println(findLinks(url))
links, err := findLinks(url)
log.Println(links, err)
We ll-chosennames can document the sig nificanceofafunctionsresults. Names arepar tic u-
larlyvaluablewhenafunctionretur nsmultipleresults ofthe sametyp e,like
func Size(rect image.Rectangle) (width, height int)
func Split(path string) (dir, file string)
func HourMinSec(t time.Time) (hour, minute, second int)
butitsnot always necessary tonamemultipleresults solelyfor documentation.For ins tance,
conv ent ion dic tates thatanal bool resu ltindic ates success; an error resu ltoften needsno
explanation.
In a functionwit h name d resu lts, the operands ofaretur n st atement may beomitt ed. Thisis
called a bare return.
// CountWordsAndImages does an HTTP GET request for the HTML
// document url and returns the number of words and images in it.
func CountWordsAndImages(url string) (words, images int, err error) {
resp, err := http.Get(url)
if err != nil {
return
}
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 5.4. ERRORS 127
doc, err := html.Parse(resp.Body)
resp.Body.Close()
if err != nil {
err = fmt.Errorf("parsing HTML: %s", err)
return
}
words, images = countWordsAndImages(doc)
return
}
func countWordsAndImages(n *html.Node) (words, images int) { /* ... */ }
Abareretur n is a shorthand way toretur n each ofthe named resultvar iables in order,soin
thefunctionabove , each retur n st atement isequivalentto
return words, images, err
In functions likethisone,wit h many retur n st atementsand several results, bareretur nscan
re duce code dup lic ation, but the y rarelymakecodeeasier tounderst and.For ins tance,itsnot
obviou s at rs t glance thatthe two early retur nsare equivalentto return 0, 0, err (b ecaus e
theresultvar iables words and images areinitialize d to their zerovalues) andthatthe nal
return is equivalentto return words, images, nil.For thisreason, bareretur nsare best
us edsparing ly.
Exercis e 5.5: Implement countWordsAndImages.(SeeExercis e 4.9 for word-split ting.)
Exercis e 5.6: Mo dif y the corner func tionin gopl.io/ch3/surface (§3.2) touse named
resu lts andabareretur n st atement.
5.4. Errors
Some functions always succe e d at their task. For example, strings.Contains and str-
conv.FormatBool have well-define d resu lts for all possibleargumentvalues andcannot fai l
barring cat astrop hic andunp redic table scenar ios li kerunningout of memor y,where the
sy mpt omisfar fro m thecause and fro m whichthereslit tle hop e of recov ery.
Ot her functions always succe e d so lon g as their pre con dit ion s aremet. For example, the
time.Date func tionalways cons tructsatime.Time from itscomponentsye ar, month,and
so onun lessthe lastargument(thetimezon e)is nil,inwhichcas e it panics. Thispanic isa
sure sig n of a bug in the cal lingcodeand shouldnever happen in a wel l-w rit ten program.
Fo r many other functions,eveninawel l-w rit ten program, successisnot assure d becaus e it
dep ends onfac tor s beyond the programmerscontrol . AnyfunctionthatdoesI/O,for exam-
ple, mustconfrontthe possibi lit y of error,and onlyanve programmer belie ves a simpleread
or write cannot fai l.Indeed,itswhenthe most reliableoperat ions fai l unexp ectedlythatwe
most need toknowwhy.
Er ror s arethu s an importantpar t of a packagesAPI oranapp lic ationsuserint erface,and fai l-
ureisjustone ofseveral expec ted beh avior s.Thisisthe appro ach Gotakes toerror handling.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
128 CHAPTER 5. FUNCTIONS
Afunctionfor whichfai lureisanexp ected beh avior retur nsanaddition alresult, conv ent ion-
al lythe lastone.Ifthe fai lurehas onlyone possiblecause,the resultisaboole an, usually
called ok,asinthisexampleofacache looku p that always succe e dsunlessthere was noent ry
forthatkey :
value, ok := cache.Lookup(key)
if !ok {
// ...cache[key] does not exist...
}
Mo reoften, andesp eci ally for I/O,the fai luremay haveavar ietyofcausesfor whichthe cal ler
wi l l ne e d an explanation.Insuchcas es, thetyp e of the addition alresultis error.
Thebui lt-in typ e error is anint erface typ e.Well see moreofwhatthismeans and its impli-
cation s forerror handlinginChapt er7.For now itsenoug h to knowthatan error maybenil
or non-ni l,thatnil imp lies successand non-ni l implies failure , andthatanon-ni l error has
an erro r mess age str ing whichwecan obt ain bycal lingits Error method orprint bycal ling
fmt.Println(err) or fmt.Printf("%v", err).
Us ual lywhenafunctionretur nsa non-ni l er ror,its other results areundefine d andshouldbe
ig nored.How ever, a few functions may retur n partialresults in erro r cases. For example, ifan
er ror occ urswhi le re adingfro m a file,acal l to Read returnsthe numberofbytes itwas able to
re ad an d an error value descr ibing the pro blem. For cor rec t behavior,som e callersmay need
to pro cessthe incompletedat a before handlingthe error,soitisimp ortantthatsuchfunctions
clearlydocumenttheir results.
Gosappro ach sets itapart from manyother langu ages in whichfai lures arerep orted using
ex cepti ons,not ordinar y values. Alt hough Godoeshaveanexception mechanism ofsor ts, as
we will see inSec tion 5.9, itisusedonlyfor rep ortingtruly unexp ected error s that indic atea
bug, not the routine error s that a robustprogram shouldbebui lttoexp ect.
Thereasonfor thisdesig n is thatexception s tend toent ang lethe des crip tionofanerror wit h
thecontrol ow required tohandleit, often leadingtoanundesirable out com e:routine error s
arerep orted tothe end userinthe for m of anincomprehensible stack trace,full of
infor mat ionabout the str uctureofthe program but lacking int ellig iblecontext aboutwhat
went wro ng.
By contrast, Goprogramsuse ordinar y cont rol-flow mechanismslike if and return to
resp ond toerror s.Thissty leundeni ably demands thatmoreatt ent ion bepaid toerror-han-
dlinglog ic, butthatispre cis ely the point.
5.4.1. Error-Handling Strategies
Wh enafunctioncal l returnsanerror,itsthe cal lersresponsibilit y to checkitand take
appropriateaction. Dep endingonthe sit uat ion, there may beanumb erofpossibi lit ies. Lets
take a lookatfive ofthem.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 5.4. ERRORS 129
First,and most common, isto prop agate theerror,sothatafai lureinasubro utine becom esa
fai lureofthe cal lingroutine.Wesaw examples ofthisinthe findLinks func tionof
Section5.3. Ifthe cal l to http.Get fai ls, findLinks returnsthe HTTPerror tothe cal ler
withoutfur therado:
resp, err := http.Get(url)
if err != nil {
return nil, err
}
In contrast, if the cal l to html.Parse fai ls, findLinks do es notretur n theHTMLparsers
er ror direc tly because itlacks two cruci al pieces ofinfor mat ion: thatthe error occ urre d in the
pars er, and the URL ofthe documentthatwas beingparsed. Inthiscas e, findLinks con-
st ruc ts anew error message thatincludes bot h pieces ofinfor mat ionaswel l as the underly ing
pars e er ror :
doc, err := html.Parse(resp.Body)
resp.Body.Close()
if err != nil {
return nil, fmt.Errorf("parsing %s as HTML: %v", url, err)
}
The fmt.Errorf func tionfor mats an erro r mess age using fmt.Sprintf andretur nsanew
error value.Weuse ittobui ld des crip tiveerror s by successive lyprefixingaddition alcontext
infor mat iontothe originalerror message . Wh enthe error isult imate lyhandled bythe
prog rams main func tion, itshouldprovide a cle ar caus alchain fro m therootpro blem tothe
ov eral l fai lure, reminiscentofaNASAaccidentinv est igat ion:
genesis: crashed: no parachute: G-switch failed: bad relay orientation
Becaus e er ror messagesare fre quentlychained toget her,message str ingsshouldnot becapit al-
ize d andnewlines shouldbeavoide d.The resulting error s maybelon g,but the y wi l l be self-
cont ained whenfound bytools like grep.
Wh endesig ningerror messages, bedelib erate, sothateachone isameaningf ul des crip tionof
thepro blem wit h sufficientand relevantdet ail, and becon sistent,sothaterror s returned by
thesamefunctionorbyagro upoffunctions inthe samepackageare simi lar in for m andcan
be dealt wit h in the sameway.
Fo r example, the os packageguarante esthatevery error retur ned byafile operat ion, suchas
os.Open or the Read, Write,or Close methodsofanopen file,des crib esnot justthe natureof
thefai lure(permissiondenie d,nosuchdirec tor y,and soon) but als o thenameofthe file,so
thecal ler ne e dntinclude thisinfor mat ion in the error message itcon str ucts.
In general,the cal l f(x) is responsible for rep ortingthe att emp ted operat ion f andthe argu-
ment value x as the y re latetothe context ofthe error.The cal ler is responsible for addingfur-
ther infor mat ionthatithas but the cal l f(x) do es not, suchasthe URL in the cal l to
html.Parse ab ove .
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
130 CHAPTER 5. FUNCTIONS
Letsmov e on tothe secon d st rateg y forhandlingerror s.For error s that represent transientor
unpredic table pro blems, itmay makesense to re try thefai le d op erat ion, possibly wit h adel ay
betweentries, andperhaps wit h alimitonthe numberofatt emp tsorthe timespent trying
before giv ing upent ire ly.
gopl.io/ch5/wait
// WaitForServer attempts to contact the server of a URL.
// It tries for one minute using exponential back-off.
// It reports an error if all attempts fail.
func WaitForServer(url string) error {
const timeout = 1 * time.Minute
deadline := time.Now().Add(timeout)
for tries := 0; time.Now().Before(deadline); tries++ {
_, err := http.Head(url)
if err == nil {
return nil // success
}
log.Printf("server not responding (%s); retrying...", err)
time.Sleep(time.Second << uint(tries)) // exponential back-off
}
return fmt.Errorf("server %s failed to respond after %s", url, timeout)
}
Third, ifprogressisimp ossible,the cal ler canprint the error and stopthe program gracef ully,
butthiscours e of actionshouldgeneral lyberes erve d forthe main packageofaprogram.
Librar y func tions shouldusu allypro pagateerror s to the cal ler,unlessthe error isasig n of an
internal inconsistenc ythat is, a bug .
// (In function main.)
if err := WaitForServer(url); err != nil {
fmt.Fprintf(os.Stderr, "Site is down: %v\n", err)
os.Exit(1)
}
Amoreconvenientway toachie vethe sameeffec t is tocal l log.Fatalf.Aswit h al l the log
func tions,bydefau ltitprexesthe timeand datetothe error message .
if err := WaitForServer(url); err != nil {
log.Fatalf("Site is down: %v\n", err)
}
Thedefau ltfor mat ishelpf ul in a lon g-r unningser ver,but lesssofor anint erac tivetool:
2006/01/02 15:04:05 Site is down: no such domain: bad.gopl.io
Fo r amoreatt rac tiveout put,wecan set the prexusedbythe log packagetothe nameofthe
command, and sup pressthe displ ayofthe dateand time:
log.SetPrefix("wait: ")
log.SetFlags(0)
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 5.4. ERRORS 131
Fo urth, in som e cases, itssufficientjusttolog the error and thencontinue, perhaps wit h
re duce d func tionality.Again theresachoice bet weenusingthe log package, whichaddsthe
usualprefix:
if err := Ping(); err != nil {
log.Printf("ping failed: %v; networking disabled", err)
}
andprint ing direc tly tothe stand ard error stream:
if err := Ping(); err != nil {
fmt.Fprintf(os.Stderr, "ping failed: %v; networking disabled\n", err)
}
(A ll log func tions app end a newline if one isnot already present.)
Andfifth and nally, inrarecas es we can safelyignore an error ent ire ly:
dir, err := ioutil.TempDir("", "scratch")
if err != nil {
return fmt.Errorf("failed to create temp dir: %v", err)
}
// ...use temp dir...
os.RemoveAll(dir) // ignore errors; $TMPDIR is cleaned periodically
Thecal l to os.RemoveAll mayfai l,but the program ignoresitbecause the operat ingsystem
periodic ally cle ansout the temporar y direc tor y.Inthiscas e,dis cardingthe error was inten-
tion al, but the program logic wou ldbethe samehad wefor g ott entodealwit h it.Get into the
habit of con sider ingerror s af ter every functioncal l,and whenyou delib eratelyignoreone,
do cumentyourint ent ion cle arly.
Er ror handlinginGohas a par tic ularrhythm. After che cking anerror,fai lureisusu allydealt
with beforesuccess. Iffai lurecausesthe functiontoretur n, thelog ic forsuccessisnot
indente d within an else blockbut fol lowsatthe out erlevel.Functions tendtoexhibit a com-
monstr ucture, wit h aser ies of initial che cks torej e cterror s,fol low edbythe subst anceofthe
func tion at the end,minimal lyindente d.
5.4.2. End ofFile (EOF)
Us ual ly, the var ietyoferror s that a functionmay retur n is int erest ing tothe end userbut not
to the int erveningprogram logic. On occasion, how ever, a program musttakedif ferent
ac tions dep endingonthe kindoferror thathas occ urre d.Con sider an attemp t to read n bytes
of dat a from a file.If n is chosentobethe lengt h of the file,any error representsafai lure. On
theother hand, ifthe cal ler repeatedlytries toreadfixe d-size chunksunt i l thefile isexhausted,
thecal ler mu strespond dif ferentlytoanend-of-file condit ion thanitdoestoall other erro rs.
Fo r this reason, the io packageguarante esthatany readfai lurecausedbyanend-of-file condi-
tion isalways rep orted byadistinguishe d er ror, io.EOF,whichisdefine d as fol lows:
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
132 CHAPTER 5. FUNCTIONS
package io
import "errors"
// EOF is the error returned by Read when no more input is available.
var EOF = errors.New("EOF")
Thecal ler candetec t this con dit ion usingasimplecomparison, as in the loopbelow,which
re ads runes fro m thestand ard inp ut. (The charcount prog ram in Sec tion 4.3 provides a more
comp leteexample.)
in := bufio.NewReader(os.Stdin)
for {
r, _, err := in.ReadRune()
if err == io.EOF {
break // finished reading
}
if err != nil {
return fmt.Errorf("read failed: %v", err)
}
// ...use r...
}
Sinceinanend-of-file condit ion there isnoinfor mat iontorep ort besides the fac t of it, io.EOF
hasaxe d er ror message , "EOF".For other erro rs, wemay need torep ort bot h thequality and
qu ant ity ofthe error,sotospeak, soaxe d er ror value will not do. InSec tion 7.11, well
pres entamoresystematicway todistinguish cer tain erro r values fro m ot hers.
5.5. Function Values
Func tions are rst-class values in Go: like other values, functionvalues havetyp es, andthe y
maybeassig ned tovar iables orpassedtoorretur ned fro m func tions.Afunc tionvalue may
be cal le d li keany other function. For example:
func square(n int) int {return n * n }
func negative(n int) int {return -n }
func product(m, n int) int { return m * n }
f:=square
fmt.Println(f(3)) // "9"
f=negative
fmt.Println(f(3)) // "-3"
fmt.Printf("%T\n", f) // "func(int) int"
f=product // compile error: can'tassign f(int, int) int to f(int) int
Thezerovalue ofafunctiontyp e is nil.Cal linganil functionvalue causesapanic:
var f func(int) int
f(3) // panic: call of nil function
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 5.5. FUNCTION VALUES 133
Func tionvalues may becompare d with nil:
var f func(int) int
if f != nil {
f(3)
}
butthe y arenot comparable,sothe y maynot becompare d against eachother orusedaskeys
in a map.
Func tionvalues let usparameter ize ourfunctions overnot justdat a, butbeh avior too.The
st and ard librar ies cont ain manyexamples. For ins tance, strings.Map applies a functionto
each charac ter ofastr ing , joiningthe results tomakeanother str ing .
func add1(r rune) rune { return r + 1 }
fmt.Println(strings.Map(add1, "HAL-9000")) // "IBM.:111"
fmt.Println(strings.Map(add1, "VMS")) // "WNT"
fmt.Println(strings.Map(add1, "Admix")) // "Benjy"
The findLinks func tionfro m Section5.2 usesahelperfunction, visit,tovisit all the nodes
in an HTML document and app l y an action toeachone.Usingafunctionvalue,wecan sep a-
ratethe log ic fortre e traversalfro m thelog ic forthe actiontobeapp lie d to eachnode, letting
us reuse the traversalwit h dif ferentactions.
gopl.io/ch5/outline2
// forEachNode calls the functions pre(x) and post(x) for each node
// x in the tree rooted at n. Both functions are optional.
// pre is called before the children are visited (preorder) and
// post is called after (postorder).
func forEachNode(n *html.Node, pre, post func(n *html.Node)) {
if pre != nil {
pre(n)
}
for c := n.FirstChild; c != nil; c = c.NextSibling {
forEachNode(c, pre, post)
}
if post != nil {
post(n)
}
}
The forEachNode func tionaccepts two functionarguments, one tocal l before a nodeschi l-
dren arevisit edand one tocal l af ter.Thisarrangement gives the cal ler agre atdealof flexi-
bi lit y.For example, the functions startElement and endElement pr int the start and end tags
of anHTMLelementlike <b>...</b>:
var depth int
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
134 CHAPTER 5. FUNCTIONS
func startElement(n *html.Node) {
if n.Type == html.ElementNode {
fmt.Printf("%*s<%s>\n", depth*2, "", n.Data)
depth++
}
}
func endElement(n *html.Node) {
if n.Type == html.ElementNode {
depth--
fmt.Printf("%*s</%s>\n", depth*2, "", n.Data)
}
}
Thefunctions als o indentthe out put usinganother fmt.Printf tr ick.The * adverb in %*s
pr intsastr ing padde d with a var iable numberofspaces. Thewidth and the str ing are
prov ide d by the arguments depth*2 and "".
If wecal l forEachNode on anHTMLdocument, like this:
forEachNode(doc, startElement, endElement)
we getamoreelaboratevar iat iononthe out put of our earlier outline prog ram:
$gobuild gopl.io/ch5/outline2
$./outline2 http://gopl.io
<html>
<head>
<meta>
</meta>
<title>
</title>
<style>
</style>
</head>
<body>
<table>
<tbody>
<tr>
<td>
<a>
<img>
</img>
...
Exercis e 5.7: De velop startElement and endElement into a general HTML prett y-print er.
Pr int comment nodes, text nodes, andthe att ribut esofeachelement(<a href='...'>). Use
short for mslike <img/> insteadof <img></img> when an elementhas nochi ldren. Write a
test toens ure thatthe out put can beparsedsuccessf ully. (SeeChapt er11.)
Exercis e 5.8: Mo dif y forEachNode so thatthe pre and post func tions retur n aboole an resu lt
indic atingwhether tocontinuethe traversal. Use ittowrite a function ElementByID with the
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 5.6. ANONYMOUS FUNCTIONS 135
fo llowing sig naturethatfind s thefirs t HTML elementwit h thespecified id attr ibute. The
func tionshouldstopthe traversal as soon as a match isfound.
func ElementByID(doc *html.Node, id string) *html.Node
Exercis e 5.9: Wr ite a function expand(s string, f func(string) string) string that
repl aces eachsubst ring ‘‘$foo’’ within s by the text retur ned by f("foo").
5.6. Anonymous Functions
Name d func tions can bedeclare d on lyatthe packagelevel,but wecan use a func tionlit era l to
denot e afunctionvalue wit hin anyexpression. A func tionlit eral iswritt enlikeafunction
de clarat ion, but wit houtanamefol low ing the func ke yword . It isanexpression, andits value
is cal le d an an ony mou s func tion.
Func tionlit eralslet usdefine a functionatits point ofuse.Asanexample, the earlier cal l to
strings.Map canberewritt enas
strings.Map(func(r rune) rune { return r + 1 }, "HAL-9000")
Mo reimp ortantly, functions define d in thisway haveaccesstothe ent ire lexic al enviro nment,
so the inner functioncan refer tovar iables fro m theenclosingfunction, as thisexample
shows:
gopl.io/ch5/squares
// squares returns a function that returns
// the next square number each time it is called.
func squares() func() int {
var x int
return func() int {
x++
return x * x
}
}
func main() {
f:=squares()
fmt.Println(f()) // "1"
fmt.Println(f()) // "4"
fmt.Println(f()) // "9"
fmt.Println(f()) // "16"
}
Thefunction squares returnsanother function, oftyp e func() int.Acall to squares cre-
ates a localvar iable x andretur nsananony mou s func tionthat, eachtimeitiscal le d,incre-
ments x andretur nsits squ are . Asecon d call to squares wouldcre ate a secon d var iable x and
return a new anony mou s func tionwhichincrementsthatvar iable.
The squares exampledemon strates thatfunctionvalues arenot justcodebut can havestate.
Theanony mou s inner functioncan accessand updatethe local variables ofthe enclosing
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
136 CHAPTER 5. FUNCTIONS
func tion squares.These hidden variablereferences arewhy weclassif y func tions asreference
typesand why functionvalues arenot comparable.Functionvalues like these are imp le-
ment edusingatechnique cal le d cl osure s,and Goprogrammersoften use thister m forfunc-
tion values.
Here again wesee anexamplewhere the lifet imeofavar iable isnot deter mined byits scope:
thevar iable x existsafter squares hasretur ned wit hin main,eventhoug h x is hidden inside f.
As a som ewhat academic exampleofanony mou s func tions,con sider the pro blem ofcom-
puting a sequence ofcomputerscience coursesthatsat isfies the prere quisite requirements of
each one.The prere quisitesare given in the prereqs tablebelow,whichisamapping fro m
each cours e to the listofcours esthatmustbecompleted beforeit.
gopl.io/ch5/toposort
// prereqs maps computer science courses to their prerequisites.
var prereqs = map[string][]string{
"algorithms": {"data structures"},
"calculus": {"linear algebra"},
"compilers": {
"data structures",
"formal languages",
"computer organization",
},
"data structures": {"discrete math"},
"databases": {"data structures"},
"discrete math": {"intro to programming"},
"formal languages": {"discrete math"},
"networks": {"operating systems"},
"operating systems": {"data structures", "computer organization"},
"programming languages": {"data structures", "computer organization"},
}
Thiskindofpro blem isknown astop olog ical sor ting. Con ceptu ally, the prere quisite
infor mat ionfor msadirec ted graphwit h anodefor eachcours e andedges fro m each cours e to
thecours esthatitdep ends on. Thegraphisacyclic: there isnopat h from a cours e that leads
back to its elf.Wecan comp ute a valid sequence usingdepth-firs t search through the graph
with thecodebelow :
func main() {
for i, course := range topoSort(prereqs) {
fmt.Printf("%d:\t%s\n", i+1, course)
}
}
func topoSort(m map[string][]string) []string {
var order []string
seen := make(map[string]bool)
var visitAll func(items []string)
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 5.6. ANONYMOUS FUNCTIONS 137
visitAll = func(items []string) {
for _, item := range items {
if !seen[item] {
seen[item] = true
visitAll(m[item])
order = append(order, item)
}
}
}
var keys []string
for key := range m {
keys = append(keys, key)
}
sort.Strings(keys)
visitAll(keys)
return order
}
Wh enananony mou s func tionrequires rec ursion, as in thisexample, wemustfirs t de clare a
var iable,and thenassig n theanony mou s func tiontothatvar iable.Had these two steps been
combined inthe declarat ion, the functionlit eral wou ldnot bewit hin the scope ofthe var iable
visitAll so itwou ldhavenoway tocal l itself rec ursively:
visitAll := func(items []string) {
// ...
visitAll(m[item]) // compile error: undefined: visitAll
// ...
}
Theout put of the toposort prog ram is shown below.Itisdeter minist ic, an often-desirable
prop ertythatdoesntalways come for fre e.Here, the values ofthe prereqs mapare slices, not
more maps, sotheir iterat ionorder isdeter minist ic, andwesor ted the keysof prereqs before
making the initial cal lsto visitAll.
1: intro to programming
2: discrete math
3: data structures
4: algorithms
5: linear algebra
6: calculus
7: formal languages
8: computer organization
9: compilers
10: databases
11: operating systems
12: networks
13: programming languages
Letsretur n to our findLinks example. Weve mov edthe lin k-ext raction function
links.Extract to its own package, since well use itagain in Chapt er8.Wereplace d the
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
138 CHAPTER 5. FUNCTIONS
visit func tionwit h an anonymou s func tionthatapp end s to the links slice direc tly,and used
forEachNode to handlethe traversal. Since Extract ne e dsonlythe pre func tion, itpasses
nil forthe post argument.
gopl.io/ch5/links
// Package links provides a link-extraction function.
package links
import (
"fmt"
"net/http"
"golang.org/x/net/html"
)
// Extract makes an HTTP GET request to the specified URL, parses
// the response as HTML, and returns the links in the HTML document.
func Extract(url string) ([]string, error) {
resp, err := http.Get(url)
if err != nil {
return nil, err
}
if resp.StatusCode != http.StatusOK {
resp.Body.Close()
return nil, fmt.Errorf("getting %s: %s", url, resp.Status)
}
doc, err := html.Parse(resp.Body)
resp.Body.Close()
if err != nil {
return nil, fmt.Errorf("parsing %s as HTML: %v", url, err)
}
var links []string
visitNode := func(n *html.Node) {
if n.Type == html.ElementNode && n.Data == "a" {
for _, a := range n.Attr {
if a.Key != "href" {
continue
}
link, err := resp.Request.URL.Parse(a.Val)
if err != nil {
continue // ignore bad URLs
}
links = append(links, link.String())
}
}
}
forEachNode(doc, visitNode, nil)
return links, nil
}
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 5.6. ANONYMOUS FUNCTIONS 139
In ste adofapp endingthe raw href attr ibute value tothe links slice,thisversionparsesitasa
URL rel ative tothe bas e URL ofthe document, resp.Request.URL.The resulting link is in
abs olut e form,suitablefor use inacal l to http.Get.
Craw lingthe web is,atits heart,aproblem ofgraphtraversal. The topoSort exampleshowe d
adepth-firs t traversal; for our web craw ler,well use bre adt h-firs t traversal, atleast initial ly. In
Chapter 8, well explore con cur renttraversal.
Thefunctionbelow enc apsulates the ess enceofabre adt h-firs t traversal. The cal ler prov ides
an initial list worklist of itemstovisit and a functionvalue f to cal l foreachitem. Eachitem
is identied byastr ing . Thefunction f returnsalistofnew items toapp end tothe wor klist.
The breadthFirst func tionretur nswhenall items havebeenvisit ed. Itmaintainsaset of
st rings toens ure thatnoitemisvisit edtwice.
gopl.io/ch5/findlinks3
// breadthFirst calls f for each item in the worklist.
// Any items returned by f are added to the worklist.
// f is called at most once for each item.
func breadthFirst(f func(item string) []string, worklist []string) {
seen := make(map[string]bool)
for len(worklist) > 0 {
items := worklist
worklist = nil
for _, item := range items {
if !seen[item] {
seen[item] = true
worklist = append(worklist, f(item)...)
}
}
}
}
As weexplained inpassinginChapt er3,the argument ‘‘f(item)...’’ caus esall the items in
thelistretur ned by f to beapp ended tothe wor klist.
In our crawler,items are URLs. The crawl func tionwell sup ply to breadthFirst pr intsthe
URL, extrac ts itslin ks, andretur nsthemsothatthe y to o arevisit ed.
func crawl(url string) []string {
fmt.Println(url)
list, err := links.Extract(url)
if err != nil {
log.Print(err)
}
return list
}
To start the craw ler of f,well use the command-linearguments as the initial URLs.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
140 CHAPTER 5. FUNCTIONS
func main() {
// Crawl the web breadth-first,
// starting from the command-line arguments.
breadthFirst(crawl, os.Args[1:])
}
Letscraw l theweb startingfro m https://golang.org.Hereare som e of the resulting lin ks:
$gobuild gopl.io/ch5/findlinks3
$./findlinks3 https://golang.org
https://golang.org/
https://golang.org/doc/
https://golang.org/pkg/
https://golang.org/project/
https://code.google.com/p/go-tour/
https://golang.org/doc/code.html
https://www.youtube.com/watch?v=XCsL89YtqCs
http://research.swtch.com/gotour
https://vimeo.com/53221560
...
Thepro cessend s when allreach ableweb pages havebeencraw ledorthe memory ofthe com-
puterisexhausted.
Exercis e 5.10: Re writ e topoSort to use maps insteadofslices andeliminatethe initial sor t.
Verify thatthe results, thoug h nondeter minist ic, arevalid top olog ical order ings.
Exercis e 5.11: Theins tructor of the linearalgebra course decides thatcalc ulu s is now a
prerequisite. Extendthe topoSort func tiontorep ort cyc les.
Exercis e 5.12: The startElement and endElement func tions in gopl.io/ch5/outline2
(§5.5) share a globalvar iable, depth.Tur n them into anony mou s func tions thatshare a var i-
able localtothe outline func tion.
Exercis e 5.13: Mo dif y crawl to makelocal copies ofthe pages it nd s,cre ating direc tor ies as
ne cessary.Dontmakecopies ofpages thatcom e from a dif ferentdom ain. For example, ifthe
or iginalpagecom esfro m golang.org,saveall files fro m there, but exc lude onesfro m
vimeo.com.
Exercis e 5.14: Us e the breadthFirst func tiontoexplore a dif ferentstr ucture. For example,
youcou lduse the cours e dep endencies fro m the topoSort example(adirec ted graph), the file
systemhierarchyonyourcomputer(atre e), oralistofbus orsubway routesdow nlo ade d from
your city gov ernmentsweb sit e (an undirec ted graph).
5.6.1. Caveat: Capturing Iteration Variables
In thissec tion,well lookatapit fal l of Goslexic al scop e rules thatcan cause sur prisingresults.
We urge youtounderst and the pro blem beforepro ceeding, because the trap can ensnareeven
exp erience d prog rammers.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 5.6. ANONYMOUS FUNCTIONS 141
Consider a program thatmustcre ate a set ofdirec tor ies andlater remov e them. Wecan use a
slice offunctionvalues toholdthe cle an-upoperat ions.(Fo r brevit y,wehaveomitt edall error
hand ling in thisexample.)
var rmdirs []func()
for _, d := range tempDirs() {
dir := d // NOTE: necessary!
os.MkdirAll(dir, 0755) // creates parent directories too
rmdirs = append(rmdirs, func() {
os.RemoveAll(dir)
})
}
// ...do some work...
for _, rmdir := range rmdirs {
rmdir() // clean up
}
Yo u maybewon der ingwhy weassig ned the loopvar iable d to a new local variable dir within
theloopbody, ins teadofjustnamingthe loopvar iable dir as in thissubtlyincor rec t var iant:
var rmdirs []func()
for _, dir := range tempDirs() {
os.MkdirAll(dir, 0755)
rmdirs = append(rmdirs, func() {
os.RemoveAll(dir) // NOTE: incorrect!
})
}
Thereasonisacon sequence ofthe scope rules for loopvar iables. Inthe program immediate ly
ab ove , the for lo opint roduces a new lexic al blockinwhichthe var iable dir is declare d.All
func tionvalues create d by thisloop ‘‘capt ure’’ andshare the samevar iablean addressable
storagelocat ion, not its value at thatpar tic ularmom ent.The value of dir is updated insuc-
cessive iterat ions,sobythe timethe cle anup functions are cal le d,the dir var iable has been
up dated several times bythe now-comp leted for lo op. Thu s dir holdsthe value fro m the
naliterat ion, andcon sequentlyall cal lsto os.RemoveAll wi l l attemp t to remov e thesame
direc tor y.
Fr equently, the inner variableint roduce d to wor k around thispro blemdir in our example
is given the exac t same nameasthe out ervar iable ofwhichitisacopy, leadingtoodd-lo oking
butcruci al var iable decl arat ions likethis:
for _, dir := range tempDirs() {
dir := dir // declares inner dir, initialized to outer dir
// ...
}
Theriskisnot unique to range-b ased for lo ops. Theloop in the examplebelow suf fersfro m
thesamepro blem due tounintended capture ofthe index variable i.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
142 CHAPTER 5. FUNCTIONS
var rmdirs []func()
dirs := tempDirs()
for i := 0; i < len(dirs); i++ {
os.MkdirAll(dirs[i], 0755) // OK
rmdirs = append(rmdirs, func() {
os.RemoveAll(dirs[i]) // NOTE: incorrect!
})
}
Thepro blem ofiterat ionvar iable capture ismostoften encountere d when usingthe go st ate-
ment (Chapter8)orwit h defer (w hichwewill see inamom ent)since bot h maydel aythe
exec ution of a functionvalue until after the loophas finishe d.But the pro blem isnot inherent
to go or defer.
5.7. Variadic Functions
A variadic fun cti on is one thatcan becal le d with var yingnumbers ofarguments. Themost
fami liar examples are fmt.Printf andits variants. Printf re quires one xe d argumentatthe
beginning, thenaccepts anynumberofsubsequentarguments.
To d eclare a var iadic function, the typ e of the nalparameter ispre ceded byanellipsis, ‘‘...’’,
whichindic ates thatthe functionmay becal le d with any numberofarguments ofthistyp e.
gopl.io/ch5/sum
func sum(vals ...int) int {
total := 0
for _, val := range vals {
total += val
}
return total
}
The sum func tionabove retur nsthe sum ofzeroormore int arguments. Wit hin the bodyof
thefunction, the typ e of vals is an []int slice.When sum is cal le d,any numberofvalues
maybeprovide d forits vals parameter.
fmt.Println(sum()) // "0"
fmt.Println(sum(3)) // "3"
fmt.Println(sum(1, 2, 3, 4)) // "10"
Implicitly, the cal ler al locatesanarray,copies the arguments into it, andpassesaslice ofthe
entire array tothe function. Thelastcal l ab ove thu s behavesthe sameasthe cal l below, which
shows how toinv oke a var iadic functionwhenthe arguments arealready inaslice: place an
el lipsisafter the nalargument.
values := []int{1, 2, 3, 4}
fmt.Println(sum(values...)) // "10"
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 5.8. DEFERRED FUNCTION CALLS 143
Although the ...int parameter beh aveslikea slice wit hin the functionbody, the typ e of a
var iadic functionisdistinc t from the typ e of a functionwit h an ordinar y slice parameter.
func f(...int) {}
func g([]int) {}
fmt.Printf("%T\n", f) // "func(...int)"
fmt.Printf("%T\n", g) // "func([]int)"
Variadic functions are often usedfor str ing for matting . The errorf func tionbelow con-
st ruc ts afor matte d er ror message wit h alinenumberatthe beg inning. The suffix f is a widely
fo llowe d namingconvent ion for var iadic functions thatacceptaPrintf-sty lefor mat str ing .
func errorf(linenum int, format string, args ...interface{}) {
fmt.Fprintf(os.Stderr, "Line %d: ", linenum)
fmt.Fprintf(os.Stderr, format, args...)
fmt.Fprintln(os.Stderr)
}
linenum, name := 12, "count"
errorf(linenum, "undefined: %s", name) // "Line 12: undefined: count"
The interface{} type means thatthisfunctioncan acceptany values at allfor itsfinalargu-
ments, as well explain in Chapt er7.
Exercis e 5.15: Wr ite var iadic functions max and min,analogou s to sum.Whatshouldthese
func tions dowhencal le d with noarguments? Write var iants thatrequireatleast one argu-
ment.
Exercis e 5.16: Wr ite a var iadic versionof strings.Join.
Exercis e 5.17: Wr ite a var iadic function ElementsByTagName that, given an HTML nodetre e
andzeroormorenames, retur nsall the elements thatmatch one ofthose names. Hereare two
examplecal ls:
func ElementsByTagName(doc *html.Node, name ...string) []*html.Node
images := ElementsByTagName(doc, "img")
headings := ElementsByTagName(doc, "h1", "h2", "h3", "h4")
5.8. Deferred Function Calls
Our findLinks examples usedthe out put of http.Get as the inp utto html.Parse.This
workswel l if the content ofthe requeste d URL isindeed HTML, but manypages cont ain
images, plain text, andother file for mats. Feedingsuch files into anHTMLparsercou ldhave
undesirable effe cts.
Theprogram below fetch esanHTMLdocumentand printsits tit le. The title func tion
insp ectsthe Content-Type he ader ofthe ser versrespons e andretur nsanerror ifthe doc-
umentisnot HTML.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
144 CHAPTER 5. FUNCTIONS
gopl.io/ch5/title1
func title(url string) error {
resp, err := http.Get(url)
if err != nil {
return err
}
// Check Content-Type is HTML (e.g., "text/html; charset=utf-8").
ct := resp.Header.Get("Content-Type")
if ct != "text/html" && !strings.HasPrefix(ct, "text/html;") {
resp.Body.Close()
return fmt.Errorf("%s has type %s, not text/html", url, ct)
}
doc, err := html.Parse(resp.Body)
resp.Body.Close()
if err != nil {
return fmt.Errorf("parsing %s as HTML: %v", url, err)
}
visitNode := func(n *html.Node) {
if n.Type == html.ElementNode && n.Data == "title" &&
n.FirstChild != nil {
fmt.Println(n.FirstChild.Data)
}
}
forEachNode(doc, visitNode, nil)
return nil
}
Heresatypic al session, slig htlyedite d to fit:
$gobuild gopl.io/ch5/title1
$./title1 http://gopl.io
The Go Programming Language
$./title1 https://golang.org/doc/effective_go.html
Effective Go - The Go Programming Language
$./title1 https://golang.org/doc/gopher/frontpage.png
title: https://golang.org/doc/gopher/frontpage.png
has type image/png, not text/html
Observethe dup lic ated resp.Body.Close() call,whichens uresthat title clos es thenet-
work conne ction on all exe cut ion pat hs, includingfai lures. Asfunctions growmorecomplex
andhavetohandlemoreerror s,suchdup lic ationofcle an-uplog ic maybecom e amain-
tenancepro blem. Letssee how Gosnov el defer me chanism makes things simpler.
Sy ntactic ally,adefer st atement isanordinar y func tionormet hod cal l prefixe d by the
ke yword defer.The functionand argumentexpressions are evaluate d when the statement is
exec ute d, butthe actualcal l is deferred until the functionthatcontainsthe defer st atement
has finishe d,whether nor mal ly, byexe cut ing a retur n st atement orfal lingoff the end,or
abnormal ly, bypanicking . Anynumberofcal lsmay bedefer red;the y areexe cut edinthe
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 5.8. DEFERRED FUNCTION CALLS 145
re verse ofthe order in whichthe y were defer red.
A defer st atement isoften usedwit h paired operat ions likeopenand clos e,conne ctand dis-
connec t,orlockand unlocktoens ure thatres ources arereleasedinall cas es, no matterhow
comp lex the control ow.The rig htplace for a defer st atement thatreleasesares ource is
immediate lyafter the res ource has beensuccessf ullyacquired.Inthe title func tionbelow,a
singledefer red cal l repl aces bot h previous cal lsto resp.Body.Close():
gopl.io/ch5/title2
func title(url string) error {
resp, err := http.Get(url)
if err != nil {
return err
}
defer resp.Body.Close()
ct := resp.Header.Get("Content-Type")
if ct != "text/html" && !strings.HasPrefix(ct, "text/html;") {
return fmt.Errorf("%s has type %s, not text/html", url, ct)
}
doc, err := html.Parse(resp.Body)
if err != nil {
return fmt.Errorf("parsing %s as HTML: %v", url, err)
}
// ...print doc'stitle element...
return nil
}
Thesamepattern can beusedfor other res ources beside net wor k connec tion s,for ins tance to
clos e an open file:
io/ioutil
package ioutil
func ReadFile(filename string) ([]byte, error) {
f, err := os.Open(filename)
if err != nil {
return nil, err
}
defer f.Close()
return ReadAll(f)
}
or tounlockamut ex(§9.2):
var mu sync.Mutex
var m = make(map[string]int)
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
146 CHAPTER 5. FUNCTIONS
func lookup(key string) int {
mu.Lock()
defer mu.Unlock()
return m[key]
}
The defer st atement can also beusedtopair ‘‘on ent ry’’ and ‘‘on exit’’ ac tions whendebug ging
acomplex function. The bigSlowOperation func tionbelow cal ls trace immediate ly, which
do es the ‘‘on ent ry’’ ac tionthenretur nsafunctionvalue that, whencal le d,doesthe cor-
resp onding ‘‘on exit’’ ac tion. Bydefer r ingacal l to the retur ned functioninthisway,wecan
inst rumentthe ent rypoint and all exitpointsofa functioninasinglestatement and evenpass
values, like the start time,bet weenthe two actions.But dontfor g etthe nalparenthes es in
the defer st atement,orthe ‘‘on ent ry’’ ac tionwill happenonexitand the on-exitactionwont
happ en at all!
gopl.io/ch5/trace
func bigSlowOperation() {
defer trace("bigSlowOperation")() // don'tforget the extra parentheses
// ...lots of work...
time.Sleep(10 * time.Second) // simulate slow operation by sleeping
}
func trace(msg string) func() {
start := time.Now()
log.Printf("enter %s", msg)
return func() { log.Printf("exit %s (%s)", msg, time.Since(start)) }
}
Each time bigSlowOperation is cal le d,itlogs its entr y andexitand the elaps edtimebet ween
them. (Weused time.Sleep to simulateaslowoperat ion.)
$gobuild gopl.io/ch5/trace
$./trace
2015/11/18 09:53:26 enter bigSlowOperation
2015/11/18 09:53:36 exit bigSlowOperation (10.000589217s)
Deferred functions run af ter return statementshaveupdated the functionsresultvar iables.
Becaus e an anonymou s func tioncan accessits enclosingfunctionsvar iables, includingnamed
resu lts, a defer red anony mou s func tioncan obs erve the functionsresults.
Consider the function double:
func double(x int) int {
return x + x
}
By namingits resultvar iable andaddingadefer st atement,wecan makethe functionprint its
arguments andresults eachtimeitiscal le d.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 5.8. DEFERRED FUNCTION CALLS 147
func double(x int) (result int) {
defer func() { fmt.Printf("double(%d) = %d\n", x, result) }()
return x + x
}
_=double(4)
// Output:
// "double(4) = 8"
Thistrick isoverk ill for a functionassimpleas double butmay beusefulinfunctions wit h
many retur n st atements.
Adefer red anony mou s func tioncan evenchange the values thatthe enclosingfunction
returnstoits cal ler:
func triple(x int) (result int) {
defer func() { result += x }()
return double(x)
}
fmt.Println(triple(4)) // "12"
Becaus e defer red functions arentexe cut edunt i l thever y endofafunctionsexe cut ion,a
defer st atement inaloopdes ervesext ra scrutiny.The codebelow cou ldrun out of file
des crip tor s since no file will beclos edunt i l al l files havebeenpro cessed:
for _, filename := range filenames {
f, err := os.Open(filename)
if err != nil {
return err
}
defer f.Close() // NOTE: risky; could run out of file descriptors
// ...process f...
}
Onesolut ion istomov e theloopbody, includingthe defer st atement,int o anot her function
that iscal le d on eachiterat ion.
for _, filename := range filenames {
if err := doFile(filename); err != nil {
return err
}
}
func doFile(filename string) error {
f, err := os.Open(filename)
if err != nil {
return err
}
defer f.Close()
// ...process f...
}
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
148 CHAPTER 5. FUNCTIONS
Theexamplebelow is an imp rov ed fetch prog ram (§1.5) thatwritesthe HTTPrespons e to a
lo cal file insteadoftothe stand ard out put.Itder ivesthe file namefro m thelastcomponent of
theURL pat h, whichitobt ainsusingthe path.Base func tion.
gopl.io/ch5/fetch
// Fetch downloads the URL and returns the
// name and length of the local file.
func fetch(url string) (filename string, n int64, err error) {
resp, err := http.Get(url)
if err != nil {
return "", 0, err
}
defer resp.Body.Close()
local := path.Base(resp.Request.URL.Path)
if local == "/" {
local = "index.html"
}
f, err := os.Create(local)
if err != nil {
return "", 0, err
}
n, err = io.Copy(f, resp.Body)
// Close file, but prefer error from Copy, if any.
if closeErr := f.Close(); err == nil {
err = closeErr
}
return local, n, err
}
Thedefer red cal l to resp.Body.Close shouldbefami liar bynow.Itstempt ing touse a
second defer red cal l,to f.Close,toclos e thelocal file,but thiswou ldbesubtlywro ngbecause
os.Create op ens a file for writing , creating itasneeded.Onmany file systems,not ably NFS,
wr ite error s arenot rep orted immediate lybut may bepostp one d until the file isclos ed. Fai l-
uretoche ckthe resultofthe clos e op erat ioncou ldcause seriou s data losstogounnot ice d.
Ho wever,ifbot h io.Copy and f.Close fai l,weshouldprefertorep ort the error fro m
io.Copy since itocc urre d rs t andismorelikelytotel l us the rootcause.
Exercis e 5.18: Wi thout chang ing its beh avior,rewrite the fetch func tiontouse defer to clos e
thewritable file.
5.9. Panic
Gostyp e systemcatch esmanymistakesatcompi letime, but others, like anout-of-b ounds
ar ray accessornil point erdereference,requireche cks at run time. Whenthe Goruntime
detec tsthese mistakes, it pani cs.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 5.9. PANIC 149
During a typic al panic, nor mal exec ution stops, alldefer red functioncal lsinthatgoroutine are
exec ute d, andthe program crasheswit h alog message . Thislog message includes the pani c
valu e,whichisusu allyanerror message ofsom e sort,and,for eachgoroutine,asta cktra c e
showing the stack offunctioncal lsthatwereactive at the timeofthe panic. Thislog message
of ten has enough infor mat iontodiagnos e therootcause ofthe pro blem wit houtrunningthe
prog ram again, soitshouldalways beinclude d in a bug rep ort about a panicking program.
No t al l panics come fro m theruntime.The bui lt-in panic func tionmay becal le d direc tly ; it
accepts anyvalue as an argument. A panic isoften the bestthingtodowhensom e ‘‘impossi-
ble’’ situ ation happens,for ins tance,exe cut ion reach esacas e that logic ally canthappen:
switch s := suit(drawCard()); s {
case "Spades": // ...
case "Hearts": // ...
case "Diamonds": // ...
case "Clubs": // ...
default:
panic(fmt.Sprintf("invalid suit %q", s)) // Joker?
}
Itsgood prac tice toass ert thatthe pre con dit ion s of a functionhold, but thiscan easi lybedon e
to excess. Unlessyou can provide a moreinfor mat ive error message ordetec t an erro r so oner,
thereisnopoint ass ertingacon dit ion thatthe runtime will che ckfor you .
func Reset(x *Buffer) {
if x == nil {
panic("x is nil") // unnecessary!
}
x.elements = nil
}
Although Gospanic mechanism res embles exception s in other langu ages, the sit uat ions in
whichpanic isusedare quite dif ferent. Since a panic causesthe program tocrash, itisgener-
al lyusedfor graveerror s,such as a log ical inconsistenc y in the program; diligentprogram-
mers con sider anycrashtobepro ofofabug in their code.Inarobustprogram, ‘‘exp ected’’
er ror s,the kindthatarise fro m incorrec t input, misconfigurat ion, orfai lingI/O,shouldbe
hand led gracef ully; the y arebestdealt wit h using error values.
Consider the function regexp.Compile,whichcompi les a regu lar expressionint o an efficient
form for matching. Itretur nsan error if cal le d with anill-for med patt ern,but che cking this
er ror isunnecessary and burdensome ifthe cal ler knowsthatapartic ularcal l cannot fai l.In
such cas es, itsreasonablefor the cal ler to handleanerror bypanicking , since itisbelie ved to
be imp ossible.
Sincemostregu lar expressions are lit eralsinthe program sourcecode, the regexp package
prov ides a wrapperfunction regexp.MustCompile that does thische ck:
package regexp
func Compile(expr string) (*Regexp, error) { /* ... */ }
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
150 CHAPTER 5. FUNCTIONS
func MustCompile(expr string) *Regexp {
re, err := Compile(expr)
if err != nil {
panic(err)
}
return re
}
Thewrapperfunctionmakes itconvenientfor clients toinitialize a package-le vel var iable wit h
acompi led regu lar expression, like this:
var httpSchemeRE = regexp.MustCompile(`^https?:`)//"http:" or "https:"
Of course, MustCompile shouldnot becal le d with unt ruste d inputvalues. The Must prefixisa
common namingconvent ion for functions ofthiskind, like template.Must in Sec tion 4.6.
Wh enapanic occ urs, alldefer red functions are run in reverse order,startingwit h thos e of the
topm ost functiononthe stack and pro ceedingupto main,asthe program below
demon strates:
gopl.io/ch5/defer1
func main() {
f(3)
}
func f(x int) {
fmt.Printf("f(%d)\n", x+0/x) // panics if x == 0
defer fmt.Printf("defer %d\n", x)
f(x - 1)
}
Wh enrun, the program printsthe fol low ing tothe stand ard out put:
f(3)
f(2)
f(1)
defer 1
defer 2
defer 3
Apanic occ ursdur ingthe cal l to f(0),causingthe three defer red cal lsto fmt.Printf to run.
Then the runtime ter minates the program, print ing the panic message and a stack dumpto
thestand ard error stream(simplified for clarity):
panic: runtime error: integer divide by zero
main.f(0)
src/gopl.io/ch5/defer1/defer.go:14
main.f(1)
src/gopl.io/ch5/defer1/defer.go:16
main.f(2)
src/gopl.io/ch5/defer1/defer.go:16
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 5.10. RECOVER 151
main.f(3)
src/gopl.io/ch5/defer1/defer.go:16
main.main()
src/gopl.io/ch5/defer1/defer.go:10
As wewill see soon, itispossiblefor a functiontorecov erfro m apanic sothatitdoesnot ter-
minatethe program.
Fo r di agnost icpur pos es, the runtime packagelets the programmer dumpthe stack usingthe
same machiner y.Bydefer r ingacal l to printStack in main,
gopl.io/ch5/defer2
func main() {
defer printStack()
f(3)
}
func printStack() {
var buf [4096]byte
n:=runtime.Stack(buf[:], false)
os.Stdout.Write(buf[:n])
}
thefol low ing addition altext (again simplified for clarity)isprint edtothe stand ard out put:
goroutine 1 [running]:
main.printStack()
src/gopl.io/ch5/defer2/defer.go:20
main.f(0)
src/gopl.io/ch5/defer2/defer.go:27
main.f(1)
src/gopl.io/ch5/defer2/defer.go:29
main.f(2)
src/gopl.io/ch5/defer2/defer.go:29
main.f(3)
src/gopl.io/ch5/defer2/defer.go:29
main.main()
src/gopl.io/ch5/defer2/defer.go:15
Re adersfami liar wit h exception s in other langu ages may besur prisedthat runtime.Stack
canprint infor mat ionabout functions thatseemtohavealready been ‘‘unwound.’’ Gospanic
me chanism runsthe defer red functions before it unw ind s thestack.
5.10. Recover
Gi vingupisusu allythe rig htrespons e to a panic, but not always. Itmig htbepossibleto
re cov erinsom e way,oratleast cle an up the mess beforequitt ing . Fo r example, a web ser ver
that encounters anunexp ected pro blem couldclos e theconne ction rat her thanleave the client
hang ing , anddur ingdevelopment,itmig htrep ort the error tothe clienttoo.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
152 CHAPTER 5. FUNCTIONS
If the bui lt-in recover func tioniscal le d within a defer red functionand the functioncontain-
ingthe defer st atement ispanicking , recover ends the cur rentstate ofpanic andretur nsthe
panic value.The functionthatwas panicking doesnot continuewhere itlef t of f butretur ns
normal ly. If recover is cal le d at anyother time, ithas noeffec t andretur ns nil.
To illustrate, con sider the development ofaparserfor a langu age. Evenwhenitapp earstobe
work ing well,given the complexity ofits job,bugs may still lurkinobs curecor ner cas es. We
mig htpreferthat, insteadofcrashing, the parsertur nsthese panics into ordinar y pars e er ror s,
perh aps wit h an extra message exhor tingthe userto file a bug rep ort.
func Parse(input string) (s *Syntax, err error) {
defer func() {
if p := recover(); p != nil {
err = fmt.Errorf("internal error: %v", p)
}
}()
// ...parser...
}
Thedefer red functionin Parse re cov ers fro m apanic, usingthe panic value tocon str uct an
er ror message;afancier versionmig htinclude the ent ire cal l st ack using runtime.Stack.The
defer red functionthenassig nstothe err resu lt, whichisretur ned tothe cal ler.
Re cov ering indis criminatelyfro m panics isadubious prac tice because the state ofapackages
var iables afterapanic israrelywel l define d or documente d. Perh aps a crit icalupdatetoadat a
st ruc turewas incomp lete, a file ornet wor k connec tion was opene d butnot clos ed, oralock
was acquired but not released. Fur thermore, byreplacingacrashwit h,say,aline in a log file,
indiscriminaterecov ery may cause bugs togounnot ice d.
Re cov ering fro m apanic wit hin the samepackagecan helpsimplif y thehandlingofcomplex
or unexp ected error s,but asageneral rule, you shouldnot att emp t to recov erfro m anot her
packagespanic. Public APIs shouldrep ort fai lures as errors. Simi larly,you shouldnot
re cov erfro m apanic thatmay passthrough a functionyou donot maintain, suchasacal ler-
prov ide d callback,since you cannot reasonabout itssafet y.
Fo r example, the net/http packageprovides a web ser ver thatdispatch esincomingrequests
to user-prov ide d hand ler functions.Rat her thanlet a panic in one ofthese handlerskill the
process, the ser ver cal ls recover,printsastack trace,and continues ser ving. Thisiscon-
venientinprac tice,but it doesriskleaking res ources orleaving the fai le d hand ler in an
unsp ecified state thatcou ldleadtoother pro blems.
Fo r al l theabove reasons,itssafesttorecov erselec tive lyifatall.Inother words,recov eronly
from panics thatwereint ended toberecov ere d from,whichshouldberare. Thisint ent ion
canbeencoded byusingadist inc t,unexp orted typ e forthe panic value andtesting whether
thevalue retur ned by recover hasthattyp e.(Well see one way todothisinthe next exam-
ple.)Ifso, werep ort the panic as an ordinar y error;ifnot,wecal l panic with thesamevalue
to resumethe state ofpanic.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 5.10. RECOVER 153
Theexamplebelow isavar iat iononthe title prog ram that rep ortsanerror ifthe HTML
do cumentcontainsmultiple <title> elements. Ifso, itabortsthe rec ursionbycal ling panic
with a value ofthe speci al type bailout.
gopl.io/ch5/title3
// soleTitle returns the text of the first non-empty title element
// in doc, and an error if there was not exactly one.
func soleTitle(doc *html.Node) (title string, err error) {
type bailout struct{}
defer func() {
switch p := recover(); p {
case nil:
// no panic
case bailout{}:
// "expected" panic
err = fmt.Errorf("multiple title elements")
default:
panic(p) // unexpected panic; carry on panicking
}
}()
// Bail out of recursion if we find more than one non-empty title.
forEachNode(doc, func(n *html.Node) {
if n.Type == html.ElementNode && n.Data == "title" &&
n.FirstChild != nil {
if title != "" {
panic(bailout{}) // multiple title elements
}
title = n.FirstChild.Data
}
}, nil)
if title == "" {
return "", fmt.Errorf("no title element")
}
return title, nil
}
Thedefer red handler functioncal ls recover,che cks the panic value,and rep ortsanordinar y
er ror ifthe value was bailout{}.All other non-ni l values indic ateanunexp ected panic, in
whichcas e thehandler cal ls panic with that value,undoing the effec t of recover andresum-
ingthe originalstate ofpanic. (Thisexampledoessom ewhat violate our advice aboutnot
usingpanics for ‘‘exp ected’’ er ror s,but it prov ides a comp act illustrat ionofthe mech anics.)
Fr omsom e condit ion s thereisnorecov ery.Runningout of memor y,for example, causesthe
Go runtime toter minatethe program wit h afat al er ror.
Exercis e 5.19: Us e panic and recover to write a functionthatcontainsno return st atement
yetretur nsanon-zerovalue.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
This page intentionally left blank
From the Library of YIGUANG HU
ptg16105617
6
Me thods
Sincethe early 1990s, obj e ct-oriente d prog ramming(OOP) has beenthe dominantprogram-
mingparadig m in indu str y andeducation,and nearly all widely usedlangu ages deve lop ed
since thenhaveinclude d supp ort for it.Goisnoexception.
Although there isnounivers allyaccepte d definition of obj e ct-oriente d prog ramming, for our
purpos es, an ob jec t is simply a value orvar iable thathas met hods, andametho d is a function
asso ciate d with a par tic ulartyp e.Anobj e ct-oriente d prog ram is one thatusesmet hodsto
express the pro per ties andoperat ions ofeachdat a st ruc turesothatclients need not accessthe
objec tsrepresent ation direc tly.
In earlier chapt ers,wehavemade regu lar use ofmet hodsfro m thestand ard librar y,likethe
Seconds method oftyp e time.Duration:
const day = 24 * time.Hour
fmt.Println(day.Seconds()) // "86400"
andwedefine d amet hod ofour own inSec tion 2.5, a String method for the Celsius type:
func (c Celsius) String() string { return fmt.Sprintf("%g°C", c) }
In thischapt er, the rs t of two onobj e ct-oriente d prog ramming, well showhow todefine and
us e methodseffec tive ly. Well als o covertwo key princip les ofobj e ct-oriente d prog ramming,
encapsul ati on and comp ositi on.
6.1. Method Declarations
Amet hod isdeclare d with a var iantofthe ordinar y func tiondeclarat ioninwhichanext ra
parameter appearsbeforethe functionname. The parameter attach esthe functiontothe typ e
of thatparameter.
155
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
156 CHAPTER 6. METHODS
Letswrite our rs t method inasimplepackagefor plane geometr y:
gopl.io/ch6/geometry
package geometry
import "math"
type Point struct{ X, Y float64 }
// traditional function
func Distance(p, q Point) float64 {
return math.Hypot(q.X-p.X, q.Y-p.Y)
}
// same thing, but as a method of the Point type
func (p Point) Distance(q Point) float64 {
return math.Hypot(q.X-p.X, q.Y-p.Y)
}
Theext ra parameter p is cal le d themet hods re ceiver,alegac y from early obj e ct-oriente d lan-
gu ages thatdes crib edcal lingamet hod as ‘‘sendingamessage to an obj e ct.’’
In Go, wedontuse a speci al name like this or self forthe receiver; wecho ose receiver
namesjustaswewou ldfor any other parameter.Since the receivernamewill befre quently
us ed, itsagood ide a to choose somethingshort and tobecon sistent acrossmet hods. A com-
monchoice isthe rs t letterofthe typ e name,like p for Point.
In a met hod cal l,the receiverargumentapp earsbeforethe method name. Thisparal lels the
de clarat ion, in whichthe receiverparameter appearsbeforethe method name.
p:=Point{1, 2}
q:=Point{4, 6}
fmt.Println(Distance(p, q)) // "5", function call
fmt.Println(p.Distance(q)) // "5", method call
Theresnoconflic t betweenthe two declarat ions offunctions cal le d Distance ab ove . The rs t
de claresapackage-le vel functioncal le d geometry.Distance.The secon d de claresamet hod
of the typ e Point,soits nameis Point.Distance.
Theexpression p.Distance is cal le d a selector,because itselec tsthe appro priate Distance
method for the receiver p of typ e Point.Selec tor s areals o us edtoselec t elds ofstr uct typ es,
as in p.X.Since met hodsand elds inhabit the samenamespace,declaring a met hod X on the
st ruc t type Point wouldbeambiguous and the compi ler will rej e ctit.
Sinceeachtyp e hasits own namespace for met hods, wecan use the name Distance forother
methodssolon g as the y belong todif ferenttyp es. Letsdefine a typ e Path that representsa
sequence oflineseg ments andgiveitaDistance method too.
// A Path is a journey connecting the points with straight lines.
type Path []Point
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 6.1. METHOD DECLARATIONS 157
// Distance returns the distance traveled along the path.
func (path Path) Distance() float64 {
sum := 0.0
for i := range path {
if i > 0 {
sum += path[i-1].Distance(path[i])
}
}
return sum
}
Path is a name d slice typ e,not a str uct typ e li ke Point,yet wecan still define methodsfor it.
In allow ing met hodstobeass oci ated wit h anytyp e,Goisunlikemanyother obj e ct-oriente d
languages. Itisoften conv enienttodefine addition albeh avior s forsimpletyp es such asnum-
bers,str ings, slices, maps, andsom etimesevenfunctions.Met hodsmay bedeclare d on any
name d type define d in the samepackage, solon g as its underly ing typ e is neither a point ernor
an interface.
Thetwo Distance methodshavedif ferenttyp es. Theyre not rel ate d to eachother at all,
though Path.Distance us es Point.Distance internal lytocompute the lengt h of eachseg-
ment thatconne cts adjacentpoints.
Letscal l thenew met hod tocompute the per imeter ofarig httriangle:
perim := Path{
{1, 1},
{5, 1},
{5, 4},
{1, 1},
}
fmt.Println(perim.Distance()) // "12"
In the two cal lsabove tomet hodsnamed Distance,the compi ler deter mines whichfunction
to cal l basedonbot h themet hod nameand the typ e of the receiver. Inthe rs t, path[i-1]
hastyp e Point so Point.Distance is cal le d; in the secon d, perim hastyp e Path,so
Path.Distance is cal le d.
Al l methodsofagiven typ e mu sthaveunique names, but dif ferenttyp es canuse the same
name for a met hod,likethe Distance methodsfor Point and Path;theresnoneed toqualif y
func tionnames (for example, PathDistance)todis ambigu ate. Herewesee the rs t benetto
usingmet hodsoverordinar y func tions:met hod names can beshorter.The benefitismag ni-
ed for cal lsoriginating outside the package, since the y canuse the shorter name an d omit the
packagename:
import "gopl.io/ch6/geometry"
perim := geometry.Path{{1, 1}, {5, 1}, {5, 4}, {1, 1}}
fmt.Println(geometry.PathDistance(perim)) // "12", standalone function
fmt.Println(perim.Distance()) // "12", method of geometry.Path
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
158 CHAPTER 6. METHODS
6.2. Methods with a PointerReceiver
Becaus e callingafunctionmakes a copy ofeachargumentvalue,ifafunctionneedstoupdate
avar iable,orifanargumentissolarge thatwewishtoavoid copy ing it, wemustpassthe
addressofthe var iable usingapoint er. The samegoesfor met hodsthatneed toupdatethe
re ceivervar iable: weatt ach themtothe point ertyp e,suchas *Point.
func (p *Point) ScaleBy(factor float64) {
p.X *= factor
p.Y *= factor
}
Thenameofthismet hod is (*Point).ScaleBy.The parenthes es arenecessary ; without
them, the expressionwou ldbeparsedas *(Point.ScaleBy).
In a realist icprogram, conv ent ion dic tates that if any met hod of Point hasapoint erreceiver,
then al l methodsof Point shouldhaveapoint erreceiver, evenonesthatdontstr ictly need it.
Weve bro ken thisrulefor Point so thatwecan showbot h kind s of met hod.
Name d types(Point)and point ers to them(*Point)are the onlytyp es that may appear in a
re ceiverdeclarat ion. Fur thermore, to avoid ambiguities, met hod declarat ions are not per mit-
te d on named typ es that arethems elves point ertyp es:
type P *int
func (P) f() { /* ... */ } // compile error: invalid receiver type
The (*Point).ScaleBy method can becal le d by providinga*Point re ceiver, likethis:
r:=&Point{1, 2}
r.ScaleBy(2)
fmt.Println(*r) // "{2, 4}"
or this:
p:=Point{1, 2}
pptr := &p
pptr.ScaleBy(2)
fmt.Println(p) // "{2, 4}"
or this:
p:=Point{1, 2}
(&p).ScaleBy(2)
fmt.Println(p) // "{2, 4}"
Butthe lasttwo cas es areungain ly. For tunately, the langu agehelps ushere. Ifthe receiver p is
a variab le of typ e Point butthe method requires a *Point re ceiver, wecan use thisshorthand:
p.ScaleBy(2)
andthe compi ler will per for m an implicit &p on the var iable.Thiswor ksonlyfor var iables,
includingstr uct elds like p.X andarray orslice elements like perim[0].Wecannot cal l a
*Point method onanon-addressable Point re ceiver, because theresnoway toobt ain the
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 6.2. METHODS WITH A POINTER RECEIVER 159
addressofatemporar y value.
Point{1, 2}.ScaleBy(2) // compile error: can'ttake address of Point literal
Butwe can call a Point method like Point.Distance with a *Point re ceiver, because there is
away toobt ain the value fro m theaddress: justloadthe value point edtobythe receiver. The
comp iler inserts an imp licit * op erat ionfor us. Thes e twofunctioncal lsare equivalent:
pptr.Distance(q)
(*pptr).Distance(q)
Letssummar ize thes e thre e casesagain, since the y areafre quentpoint ofconfusion. Inevery
valid met hod cal l expression, exactlyone ofthese three statementsistrue.
Either the receiverargumenthas the sametyp e as the receiverparameter,for examplebot h
have typ e T or bot h have typ e *T:
Point{1, 2}.Distance(q) // Point
pptr.ScaleBy(2) // *Point
Or the receiverargumentisavariab le of typ e T andthe receiverparameter has typ e *T.The
comp iler implicitlytakes the addressofthe var iable:
p.ScaleBy(2) // implicit (&p)
Or the receiverargumenthas typ e *T andthe receiverparameter has typ e T.The compi ler
implicitlydereferences the receiver, inother words,loads the value:
pptr.Distance(q) // implicit (*pptr)
If all the methodsofanamed typ e T have a receivertyp e of T itself (not *T), itissafetocopy
inst ances ofthattyp e; callingany ofits met hodsnecessarily makes a copy.For example,
time.Duration values arelib eral lycopie d,includingasarguments tofunctions.But ifany
method has a point erreceiver, you shouldavoid copy ing ins tances of T becaus e doingsomay
viol ate int ernal invar iants. For example, copying anins tance of bytes.Buffer wouldcause
theoriginaland the copytoali as (§2.3.2) the sameunderly ing array ofbytes. Subsequent
method cal lswou ldhaveunp redic table effe cts.
6.2.1. Nil IsaValid Receiver Value
Ju stassom e func tions allow nil point ers asarguments, sodosom e methodsfor their receiver,
especi ally if nil is a meaningf ul zerovalue ofthe typ e,aswit h maps andslices. Inthissimple
lin ked listofint egers, nil repres ents the emp tylist:
// An IntList is a linked list of integers.
// A nil *IntList represents the empty list.
type IntList struct {
Value int
Tail *IntList
}
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
160 CHAPTER 6. METHODS
// Sum returns the sum of the list elements.
func (list *IntList) Sum() int {
if list == nil {
return 0
}
return list.Value + list.Tail.Sum()
}
Wh enyou define a typ e whos e methodsallow nil as a receivervalue,itswor thpoint ing this
outexplicitly in its document ation comment, as we did abov e.
Herespar t of the definition of the Values type fro m the net/url package:
net/url
package url
// Values maps a string key to a list of values.
type Values map[string][]string
// Get returns the first value associated with the given key,
// or "" if there are none.
func (v Values) Get(key string) string {
if vs := v[key]; len(vs) > 0 {
return vs[0]
}
return ""
}
// Add adds the value to key.
// It appends to any existing values associated with key.
func (v Values) Add(key, value string) {
v[key] = append(v[key], value)
}
It exp oses its represent ation asamap but als o prov ides met hodstosimplif y accesstothe map,
whos e values areslices ofstr ingsitsamu ltimap.Its clients can use its intr insic operator s
(make,slice literals, m[key],and soon), orits met hods, orbot h, as the y prefer :
gopl.io/ch6/urlvalues
m:=url.Values{"lang": {"en"}} // direct construction
m.Add("item", "1")
m.Add("item", "2")
fmt.Println(m.Get("lang")) // "en"
fmt.Println(m.Get("q")) // ""
fmt.Println(m.Get("item")) // "1" (first value)
fmt.Println(m["item"]) // "[1 2]" (direct map access)
m=nil
fmt.Println(m.Get("item")) // ""
m.Add("item", "3") // panic: assignment to entry in nil map
In the nalcal l to Get,the nil receiverbeh aveslikeanemp tymap.Wecou ldequivalentlyhave
wr itt enitas Values(nil).Get("item")),but nil.Get("item") wi l l notcompi lebecause
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 6.3. COMPOSING TYPES BYSTRUCT EMBEDDING 161
thetyp e of nil hasnot beendeter mined.Bycontrast, the nalcal l to Add panics as ittries to
up dateanil map.
Becaus e url.Values is a maptyp e andamap referstoits key/value pairsindirec tly,any
up dates anddelet ions that url.Values.Add makestothe map elements arevisible tothe
caller.How ever, aswit h ordinar y func tions,any changesamet hod makes tothe reference
itself,likesetting itto nil or mak ingitrefer toadif ferentmap dat a st ruc ture, will not be
reec ted inthe cal ler.
6.3. Composing Types byStruct Embedding
Consider the typ e ColoredPoint:
gopl.io/ch6/coloredpoint
import "image/color"
type Point struct{ X, Y float64 }
type ColoredPoint struct {
Point
Color color.RGBA
}
We cou ldhavedefine d ColoredPoint as a str uct ofthree elds,but ins teadwe embedded a
Point to provide the X and Y elds.Aswesaw in Sec tion 4.4.3, embeddinglets ustakeasyn-
tactic shortcuttodefiningaColoredPoint that cont ainsall the elds of Point,plu s some
more . If wewant, wecan selec t thefields of ColoredPoint that werecontr ibute d by the
embedde d Point withoutmentioning Point:
var cp ColoredPoint
cp.X = 1
fmt.Println(cp.Point.X) // "1"
cp.Point.Y = 2
fmt.Println(cp.Y) // "2"
Asimi lar mechanism applies tothe metho ds of Point.Wecan cal l methodsofthe emb edde d
Point eldusingareceiveroftyp e ColoredPoint,eventhoug h ColoredPoint hasno
de clare d methods:
red := color.RGBA{255, 0, 0, 255}
blue := color.RGBA{0, 0, 255, 255}
var p = ColoredPoint{Point{1, 1}, red}
var q = ColoredPoint{Point{5, 4}, blue}
fmt.Println(p.Distance(q.Point)) // "5"
p.ScaleBy(2)
q.ScaleBy(2)
fmt.Println(p.Distance(q.Point)) // "10"
Themet hodsof Point have been prom ote d to ColoredPoint.Inthisway,emb eddingallows
comp lex typ es with manymet hodstobebui ltupbythe comp ositi on of several elds,each
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
162 CHAPTER 6. METHODS
prov idingafew met hods.
Re adersfami liar wit h cl ass-b ased obj e ct-oriente d languagesmay betempt edtovie w Point as
abas e cl ass and ColoredPoint as a sub class order ive d cl ass,ortoint erpretthe rel ation ship
betweenthese typ es as if a ColoredPoint ‘‘is a’’ Point.But thatwou ldbeamistake . No tice
thecal lsto Distance ab ove . Distance hasaparameter oftyp e Point,and q is not a Point,so
although q do es have anemb edde d eldofthattyp e,wemustexplicitlyselec t it.Att emp ting
to pass q would be an error :
p.Distance(q) // compile error: cannot use q (ColoredPoint) as Point
A ColoredPoint is not a Point,but it ‘‘hasa’’ Point,and ithas two addition almet hods Dis-
tance and ScaleBy prom ote d from Point.Ifyou prefertothin k in ter msofimp lementation,
theemb edde d eldins tructsthe compi ler togenerateaddition alwrappermet hodsthatdele-
gatetothe declare d methods, equivalenttothese:
func (p ColoredPoint) Distance(q Point) float64 {
return p.Point.Distance(q)
}
func (p *ColoredPoint) ScaleBy(factor float64) {
p.Point.ScaleBy(factor)
}
Wh en Point.Distance is cal le d by the rs t of these wrappermet hods, its receivervalue is
p.Point,not p,and there isnoway for the method toaccessthe ColoredPoint in whichthe
Point is emb edde d.
Thetyp e of ananony mou s eldmay beapointer to a named typ e,inwhichcas e elds and
methodsare pro mot edindirec tly fro m thepoint ed-toobj e ct. Addinganother leve l of indi-
re ction lets usshare commonstr uctures andvar y therel ation ships bet weenobj e cts dynami-
cally. The declarat ionof ColoredPoint belowemb eds a *Point:
type ColoredPoint struct {
*Point
Color color.RGBA
}
p:=ColoredPoint{&Point{1, 1}, red}
q:=ColoredPoint{&Point{5, 4}, blue}
fmt.Println(p.Distance(*q.Point)) // "5"
q.Point = p.Point // p and q now share the same Point
p.ScaleBy(2)
fmt.Println(*p.Point, *q.Point) // "{2 2} {2 2}"
Astr uct typ e mayhavemorethanone anony mou s eld. Had wedeclare d ColoredPoint as
type ColoredPoint struct {
Point
color.RGBA
}
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 6.3. COMPOSING TYPES BYSTRUCT EMBEDDING 163
then a value ofthistyp e wouldhaveall the methodsof Point,all the methodsof RGBA,and
anyaddition almet hodsdeclare d on ColoredPoint direc tly.Whenthe compi ler res olves a
sele ctorsuchas p.ScaleBy to a met hod,itfirs t lo oks for a direc tly declare d method named
ScaleBy,thenfor met hodspro mot edoncefro m ColoredPointsemb edde d elds,thenfor
methodspro mot edtwice fro m embedde d elds wit hin Point and RGBA,and soon. Thecom-
pi ler rep ortsanerror ifthe selec tor was ambiguous because two met hodswerepro mot edfro m
thesameran k.
Methodscan bedeclare d on lyonnamed typ es (li ke Point)and point ers to them(*Point),
butthankstoemb edding, itspossibleand som etimesusefulfor unname d st ruc t typestohave
methodstoo.
Heresanice trick to illustrate. Thisexampleshows par t of a simplecache imp lemente d using
twopackage-le vel var iables, a mut ex(§9.2) andthe map thatitguard s:
var (
mu sync.Mutex // guards mapping
mapping = make(map[string]string)
)
func Lookup(key string) string {
mu.Lock()
v:=mapping[key]
mu.Unlock()
return v
}
Theversionbelow isfunctionallyequivalentbut gro ups toget her the two rel ate d var iables in a
singlepackage-le vel var iable, cache:
var cache = struct {
sync.Mutex
mapping map[string]string
}{
mapping: make(map[string]string),
}
func Lookup(key string) string {
cache.Lock()
v:=cache.mapping[key]
cache.Unlock()
return v
}
Thenew var iable gives moreexpressivenames tothe var iables rel ate d to the cache,and
becaus e the sync.Mutex eldisemb edde d within it, its Lock and Unlock methodsare
prom ote d to the unnamed str uct typ e,allow ing ustolockthe cache with a self-explanatory
sy ntax.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
164 CHAPTER 6. METHODS
6.4. Method Values and Expressions
Us ual lyweselec t andcal l amet hod inthe sameexpression, as in p.Distance(),but itspossi-
bletosep aratethese two operat ions.The selec tor p.Distance yields a metho d valu e,afunc-
tion thatbinds a met hod (Point.Distance)toaspecificreceivervalue p.Thisfunctioncan
then beinv oke d withoutareceivervalue; itneedsonlythe non-receiverarguments.
p:=Point{1, 2}
q:=Point{4, 6}
distanceFromP := p.Distance // method value
fmt.Println(distanceFromP(q)) // "5"
var origin Point // {0, 0}
fmt.Println(distanceFromP(origin)) // "2.23606797749979", ;5
scaleP := p.ScaleBy // method value
scaleP(2) // pbecomes (2, 4)
scaleP(3) // then (6, 12)
scaleP(10) // then (60, 120)
Method values areusefulwhenapackagesAPI cal lsfor a functionvalue,and the clients
desired beh avior for thatfunctionistocal l amet hod onaspecificreceiver. For example, the
func tion time.AfterFunc callsafunctionvalue afteraspecified del ay. Thisprogram usesit
to launch the rocket r af ter 10 secon ds:
type Rocket struct { /* ... */ }
func (r *Rocket) Launch() { /* ... */ }
r:=new(Rocket)
time.AfterFunc(10 * time.Second, func() { r.Launch() })
Themet hod value synt axisshorter :
time.AfterFunc(10 * time.Second, r.Launch)
Relate d to the method value isthe metho d expre ssi on.Whencal lingamet hod,asopp osedto
an ordinar y func tion, wemustsup ply the receiverinaspeci al way usingthe selec tor syntax. A
method expression, writt en T.f or (*T).f where T is a typ e,yieldsafunctionvalue wit h areg-
ular rs t parameter tak ingthe place ofthe receiver, soitcan becal le d in the usu alway.
p:=Point{1, 2}
q:=Point{4, 6}
distance := Point.Distance // method expression
fmt.Println(distance(p, q)) // "5"
fmt.Printf("%T\n", distance) // "func(Point, Point) float64"
scale := (*Point).ScaleBy
scale(&p, 2)
fmt.Println(p) // "{2 4}"
fmt.Printf("%T\n", scale) // "func(*Point, float64)"
Method expressions can behelpf ulwhenyou need a value torepresent a choice amon g several
methodsbelon gingtothe sametyp e so thatyou can cal l thechosenmet hod wit h many
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 6.5. EXAMPLE: BIT VECTOR TYPE 165
dif ferentreceivers.Inthe fol low ing example, the var iable op repres ents either the addition or
thesubtrac tionmet hod oftyp e Point,and Path.TranslateBy callsitfor eachpoint inthe
Path:
type Point struct{ X, Y float64 }
func (p Point) Add(q Point) Point { return Point{p.X + q.X, p.Y + q.Y} }
func (p Point) Sub(q Point) Point { return Point{p.X - q.X, p.Y - q.Y} }
type Path []Point
func (path Path) TranslateBy(offset Point, add bool) {
var op func(p, q Point) Point
if add {
op = Point.Add
}else {
op = Point.Sub
}
for i := range path {
// Call either path[i].Add(offset) or path[i].Sub(offset).
path[i] = op(path[i], offset)
}
}
6.5. Example: BitVector Type
Sets in Goare usu allyimp lemente d as a map[T]bool,where T is the elementtyp e.Asetrep-
resent edbyamap isver y flexiblebut,for cer tain pro blems, a speci alizedrepresent ation may
outp erfor m it.For example, indom ainssuchasdat aflow analysiswhere set elements aresmall
non-negat ive int egers,sets havemanyelements, andset operat ions likeunionand int ers ection
arecommon, a bitvec tor is ide al.
Abit vec tor usesaslice ofunsig ned int egervalues or ‘‘word s,’’ each bit of whichrepresentsa
possible elementofthe set. Theset cont ains i if the i-t h bit isset. Thefol low ing program
demon strates a simplebit vec tor typ e with thre e methods:
gopl.io/ch6/intset
// An IntSet is a set of small non-negative integers.
// Its zero value represents the empty set.
type IntSet struct {
words []uint64
}
// Has reports whether the set contains the non-negative value x.
func (s *IntSet) Has(x int) bool {
word, bit := x/64, uint(x%64)
return word < len(s.words) && s.words[word]&(1<<bit) != 0
}
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
166 CHAPTER 6. METHODS
// Add adds the non-negative value x to the set.
func (s *IntSet) Add(x int) {
word, bit := x/64, uint(x%64)
for word >= len(s.words) {
s.words = append(s.words, 0)
}
s.words[word] |= 1 << bit
}
// UnionWith sets s to the union of s and t.
func (s *IntSet) UnionWith(t *IntSet) {
for i, tword := range t.words {
if i < len(s.words) {
s.words[i] |= tword
}else {
s.words = append(s.words, tword)
}
}
}
Sinceeachwordhas 64 bits, tolocatethe bit for x,weuse the quotient x/64 as the wordindex
andthe remainder x%64 as the bit index wit hin thatword. The UnionWith op erat ionusesthe
bit w ise ORoperator | to compute the union64elements at a time. (Well revisit the choice of
64-bit words inExercis e 6.5.)
Thisimp lementation lacks manydesirable features, som e of whichare pos edasexercis es
below, but oneishardtolivewit hout: way toprint an IntSet as a str ing . LetsgiveitaString
method aswedid wit h Celsius in Sec tion 2.5:
// String returns the set as a string of the form "{1 2 3}".
func (s *IntSet) String() string {
var buf bytes.Buffer
buf.WriteByte('{')
for i, word := range s.words {
if word == 0 {
continue
}
for j := 0; j < 64; j++ {
if word&(1<<uint(j)) != 0 {
if buf.Len() > len("{") {
buf.WriteByte('')
}
fmt.Fprintf(&buf, "%d", 64*i+j)
}
}
}
buf.WriteByte('}')
return buf.String()
}
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 6.5. EXAMPLE: BIT VECTOR TYPE 167
No tice the simi lar ity ofthe String method above wit h intsToString in Sec tion 3.5.4;
bytes.Buffer is often usedthisway in String methods. The fmt packagetre ats typ es with a
String method speci ally sothatvalues ofcomplic ated typ es candispl aythems elves in a user-
friendlymanner.Ins teadofprint ing the raw represent ation of the value (a str uct inthiscas e),
fmt callsthe String method.The mech anism relies onint erfaces andtyp e assertions,which
well explain in Chapt er7.
We can now demon strate IntSet in action:
var x, y IntSet
x.Add(1)
x.Add(144)
x.Add(9)
fmt.Println(x.String()) // "{1 9 144}"
y.Add(9)
y.Add(42)
fmt.Println(y.String()) // "{9 42}"
x.UnionWith(&y)
fmt.Println(x.String()) // "{1 9 42 144}"
fmt.Println(x.Has(9), x.Has(123)) // "true false"
Awordofcaution:wedeclare d String and Has as met hodsofthe point ertyp e *IntSet not
outofnecessity,but for con sistenc y with theother two met hods, whichneed a point erreceiver
becaus e they assig n to s.words.Con sequently, an IntSet valu e do es nothaveaString
method,occ asionallyleadingtosur priseslikethis:
fmt.Println(&x) // "{1 9 42 144}"
fmt.Println(x.String()) // "{1 9 42 144}"
fmt.Println(x) // "{[4398046511618 0 65536]}"
In the rs t case,weprint an *IntSet pointer, whichdoeshaveaString method.Inthe
second cas e,wecal l String() on an IntSet var iable; the compi ler insertsthe imp licit & op er-
at ion, giv ing usapoint er, whichhas the String method.But inthe thirdcas e,because the
IntSet value does not haveaString method, fmt.Println pr intsthe represent ation of the
st ruc t instead. Itsimp ortantnot tofor g etthe & op erator.Mak ing String amet hod of
IntSet,not *IntSet,mig htbeagood ide a, butthisisacas e-by-c asejudg ment.
Exercis e 6.1: Implementthese addition almet hods:
func (*IntSet) Len() int // return the number of elements
func (*IntSet) Remove(x int) // remove x from the set
func (*IntSet) Clear() // remove all elements from the set
func (*IntSet) Copy() *IntSet // return a copy of the set
Exercis e 6.2: Dene a var iadic (*IntSet).AddAll(...int) method thatallowsalistofval-
ues tobeadde d,suchas s.AddAll(1, 2, 3).
Exercis e 6.3: (*IntSet).UnionWith comp utesthe unionoftwo sets using |,the word-p aral-
lelbit w ise ORoperator.Imp lementmet hodsfor IntersectWith, DifferenceWith,and Sym-
metricDifference forthe cor respondingset operat ions.(Thesymmetr icdif ference oftwo
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
168 CHAPTER 6. METHODS
sets cont ainsthe elements present inone set orthe other but not bot h.)
Exercis e 6.4: Addamet hod Elems that retur nsaslice cont ainingthe elements ofthe set, suit-
able for iterat ingoverwit h a range lo op.
Exercis e 6.5: Thetyp e of eachwordusedby IntSet is uint64,but 64-bit arithmeticmay be
inefficientona32-bit platfor m.Modif y theprogram touse the uint type,whichisthe most
efficientunsig ned int egertyp e forthe platfor m.Ins teadofdiv idingby64, define a con stant
holdingthe effec tive size of uint in bits, 32 or64. You can use the perhaps too-cle ver expres-
sion 32 << (^uint(0) >> 63) forthispur pos e.
6.6. Encapsulation
Avar iable ormet hod ofanobj e ctissaid tobe encapsul ate d if itisinaccessibletoclients ofthe
objec t.Enc apsulat ion, som etimescal le d in f o rmati onhiding,isakey asp ect ofobj e ct-oriente d
prog ramming.
Go has onlyone mech anism tocontrol the visibilit y of names: capit alize d identifiersare
exp orted fro m thepackageinwhichthe y aredefine d,and unc apit alize d namesare not.The
same mech anism thatlimits accesstomembers ofapackageals o limits accesstothe elds ofa
st ruc t or the methodsofatyp e.Asacon sequence,toenc apsulateanobj e ct, wemustmakeita
st ruc t.
Thatsthe reasonthe IntSet type fro m thepre vious sec tion was decl are d as a str uct typ e even
though ithas onlyasinglefield:
type IntSet struct {
words []uint64
}
We cou ldins teaddefine IntSet as a slice typ e as fol lows, thoug h of cours e wedhavetoreplace
each occ urrenceof s.words by *s in its met hods:
type IntSet []uint64
Although thisversionof IntSet wouldbeess ent ial lyequivalent, itwou ldallow clients fro m
ot her packages toreadand modif y theslice direc tly.Put another way,where asthe expression
*s couldbeusedinany package, s.words mayapp ear onlyinthe packagethatdefines
IntSet.
Anot her cons equence ofthisname-b ased mechanism isthatthe unitofenc apsulat ionisthe
package, not the typ e as in manyother langu ages. The elds ofastr uct typ e arevisible toall
co de within the samepackage. Whether the codeapp earsinafunctionoramet hod makes no
dif ference.
Encapsulat ionprovides three benefits. First,because clients cannot direc tly modif y the
objec tsvar iables, one need ins pec t fe wer statementstounderst and the possiblevalues ofthose
var iables.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 6.6. ENCAPSULATION 169
Second,hidingimp lementation det ails pre vents clients fro m dep endingonthings thatmig ht
ch ange , whichgives the desig ner gre aterfre e dom toevo l ve theimp lementation wit houtbre ak-
ingAPI compatibi lit y.
As anexample, con sider the bytes.Buffer type.Itisfre quentlyusedtoacc umulatever y
short str ings, soitisaprotableopt imizat iontores erve a lit tle extra space in the obj e ctto
avoidmemor y al location inthiscommoncas e.Since Buffer is a str uct typ e,thisspace takes
thefor m of anext ra eldoftyp e [64]byte with anunc apit alize d name.Whenthis eldwas
adde d,because itwas not exp orted,clients of Buffer outside the bytes packagewereunaware
of any change exceptimp rov edper for mance. Buffer andits Grow method are shown below,
simplified for clarity :
type Buffer struct {
buf []byte
initial [64]byte
/* ... */
}
// Grow expands the buffer'scapacity, if necessary,
// to guarantee space for another n bytes. [...]
func (b *Buffer) Grow(n int) {
if b.buf == nil {
b.buf = b.initial[:0] // use preallocated space initially
}
if len(b.buf)+n > cap(b.buf) {
buf := make([]byte, b.Len(), 2*cap(b.buf) + n)
copy(buf, b.buf)
b.buf = buf
}
}
Thethirdbenefitofenc apsulat ion, andinmanycas es themostimp ortant, isthatitpre vents
clients fro m sett ing anobj e ctsvar iables arbit rar ily.Because the obj e ctsvar iables can beset
on lybyfunctions inthe samepackage, the aut hor of thatpackagecan ensure thatall those
func tions maintain the obj e ctsint ernal invar iants. For example, the Counter type belowper-
mits clients toincrement the count erortores et it tozero, but not toset ittosom e arbit rar y
value:
type Counter struct { n int }
func (c *Counter) N() int {return c.n }
func (c *Counter) Increment() { c.n++ }
func (c *Counter) Reset() {c.n = 0 }
Func tions thatmerelyaccessormodif y internal values ofatyp e,suchasthe methodsofthe
Logger type fro m log package, below,are cal le d getters and setters.How ever, whennaminga
gett ermet hod,weusu allyomitthe Get prefix. Thispreferencefor brevit y extends toall met h-
ods, not justfieldaccessors,and toother redundant prexesaswel l,suchas Fetch, Find,and
Lookup.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
170 CHAPTER 6. METHODS
package log
type Logger struct {
flags int
prefix string
// ...
}
func (l *Logger) Flags() int
func (l *Logger) SetFlags(flag int)
func (l *Logger) Prefix() string
func (l *Logger) SetPrefix(prefix string)
Go sty ledoesnot forbidexp orted elds.Ofcours e, on ceexp orted,aeldcannot beunex-
ported wit houtanincompatiblechange tothe API,sothe initial choice shouldbedelib erate
andshouldcon sider the complexity ofthe invar iants thatmustbemaintained,the likelihood
of fut ure changes, andthe quant ity ofclientcodethatwou ldbeaffec ted byachange .
Encapsulat ionisnot always desirable.Byrevealingits represent ation asan int64 numb erof
nanosecond s, time.Duration lets ususe all the usu alarithmeticand comparisonoperat ions
with durat ions,and eventodefine con stantsofthistyp e:
const day = 24 * time.Hour
fmt.Println(day.Seconds()) // "86400"
As another example, contrast IntSet with the geometry.Path type fro m thebeg inningofthis
ch apt er. Path was define d as a slice typ e,allow ing its clients tocon str uct ins tances usingthe
slice literal synt ax, toiterateoverits pointsusingarange loop, and soon, where asthese opera-
tion s aredenie d to clients of IntSet.
Heresthe cruci al dif ference: geometry.Path is int rinsic ally a sequence ofpoints, nomoreand
no less, andwedontfores eeaddingnew elds toit, soitmakes sense for the geometry pack-
agetorevealthat Path is a slice.Incontrast, an IntSet mere lyhappens toberepresent edasa
[]uint64 slice.Itcou ldhavebeenrepresent edusing []uint,orsom ethingcompletelydif fer-
entfor sets thatare spars e or ver y smal l,and itmig htperhaps benefitfro m addition alfeatures
li ke an ext ra eldtorecordthe numberofelements in the set. For these reasons,itmakes
sens e for IntSet to beopaque.
In thischapt er, welearned how toass oci atemet hodswit h name d types, andhow tocal l thos e
methods. Alt hough met hodsare cruci al to obj e ct-oriente d prog ramming, the yre onlyhalf the
picture. Tocompleteit, weneed interfaces,the subjec t of the next chapt er.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
7
Int erfaces
Interface typ es express generalizat ions orabstrac tions about the beh avior s of other typ es. By
generalizing, int erfaces let uswrite functions thatare more flexibleand adapt ablebecause the y
arenot tie d to the det ails ofone par tic ularimp lementation.
Many obj e ct-oriente d languageshavesom e notion of int erfaces, but whatmakes Gosint er-
faces sodistinc tiveisthatthe y are sati sed imp licit ly.Inother words,theresnoneed to
de clare all the int erfaces thatagiven concrete typ e satisfies; simply possessingthe necessary
methodsisenoug h. Thisdesig n lets you cre ate new int erfaces thatare sat isfied byexist ing
concrete typ es withoutchang ing the exist ing typ es, whichispar tic ularlyusefulfor typ es
define d in packages thatyou dontcontrol .
In thischapt er, well start bylooking atthe basic mechanics ofint erface typ es andtheir values.
Alon g theway,well study several importantint erfaces fro m thestand ard librar y.ManyGo
prog ramsmakeasmuchuse ofstand ard int erfaces as the y do oftheir own ones. Final ly, well
lo okat ty peass ertions (§7.10) and ty peswitches (§7.13) andsee how the y enable a different
kind ofgenerality.
7.1. Int erfaces as Contracts
Al l thetyp es weve looke d at sofar havebeen concre tetypes.Aconcrete typ e sp ecifies the
exac t repres entation of itsvalues andexp oses the int rinsic operat ions ofthatrepresent ation,
such asarithmeticfor numbers,orindexing, append,and range forslices. A concrete typ e
mayals o prov ide addition albeh avior s thro ugh its met hods. Whenyou haveavalue ofacon-
cretetyp e,you knowexac tly whatit is andwhatyou can do with it.
Thereisanother kindoftyp e in Gocal le d an interface type.Anint erface isan ab strac t ty pe.It
do esntexp osethe represent ation or int ernal str uctureofits values, orthe set ofbasic
171
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
172 CHAPTER 7. INTERFACES
op erat ions the y supp ort;itreveals onlysom e of their met hods. Whenyou haveavalue ofan
interface typ e,you knownot hingabout whatit is;you knowonlywhatitcan do,ormore
precis ely,whatbeh avior s areprovide d by its met hods.
Throughoutthe book, weve beenusingtwo simi lar functions for str ing for matting:
fmt.Printf,whichwritesthe resulttothe stand ard out put (afile), and fmt.Sprintf,which
returnsthe resultasastring.Itwou ldbeunfor tunateifthe hardpar t, formatting the result,
hadtobedup lic ated because ofthese sup erfici al dif ferences in how the resultisused. Thanks
to int erfaces, itdoesnot.Bot h of these functions are , in effe ct, wrappers aro und a thirdfunc-
tion, fmt.Fprintf,thatisagnosticabout whathappens tothe resultitcomputes:
package fmt
func Fprintf(w io.Writer, format string, args ...interface{}) (int, error)
func Printf(format string, args ...interface{}) (int, error) {
return Fprintf(os.Stdout, format, args...)
}
func Sprintf(format string, args ...interface{}) string {
var buf bytes.Buffer
Fprintf(&buf, format, args...)
return buf.String()
}
The F prefixof Fprintf st and s for le andindic ates thatthe for matte d output shouldbe
wr itt entothe file provide d as the rs t argument. Inthe Printf case,the argument, os.Std-
out,isan *os.File.Inthe Sprintf case,how ever, the argumentisnot a file,thoug h it sup er-
fici ally res embles one: &buf is a point ertoamemor y buffer towhichbytes can bewritt en.
The rs t parameter of Fprintf is not a file either.Itsan io.Writer,whichisanint erface typ e
with thefol low ing declarat ion:
package io
// Writer is the interface that wraps the basic Write method.
type Writer interface {
// Write writes len(p) bytes from p to the underlying data stream.
// It returns the number of bytes written from p (0 <= n <= len(p))
// and any error encountered that caused the write to stop early.
// Write must return a non-nil error if it returns n < len(p).
// Write must not modify the slice data, even temporarily.
//
// Implementations must not retain p.
Write(p []byte) (n int, err error)
}
The io.Writer interface definesthe contrac t between Fprintf andits cal lers. On theone
hand,the contrac t re quires thatthe cal ler prov ide a value ofacon crete typ e li ke *os.File or
*bytes.Buffer that has a met hod cal le d Write with theappro priatesig natureand beh avior.
On the other hand, the contrac t gu arante esthat Fprintf wi l l do its job given anyvalue that
satisfies the io.Writer interface. Fprintf maynot assume thatitiswriting toafile orto
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 7.1. INTERFACES AS CONTRACTS173
memory,onlythatitcan cal l Write.
Becaus e fmt.Fprintf assumesnot hingabout the represent ation of the value andrelies only
on the beh avior s gu arante e d by the io.Writer cont rac t, we can safelypassavalue ofany con-
cretetyp e that sat isfies io.Writer as the rs t argumentto fmt.Fprintf.Thisfre e dom to
su bst itute one typ e foranother thatsat isfies the sameint erface iscal le d substitutabi lity,and is
ahal lmarkofobj e ct-oriente d prog ramming.
Letstestthisout usinganew typ e.The Write method ofthe *ByteCounter type below
mere lycountsthe bytes writt entoitbeforedis cardingthem. (Theconversionisrequired to
make the typ es of len(p) and *c match inthe += assig nmentstatement.)
gopl.io/ch7/bytecounter
type ByteCounter int
func (c *ByteCounter) Write(p []byte) (int, error) {
*c += ByteCounter(len(p)) // convert int to ByteCounter
return len(p), nil
}
Since *ByteCounter satisfies the io.Writer cont rac t, we can passitto Fprintf,whichdoes
itsstr ing for matting obliv iou s to thischange;the ByteCounter correc tly acc umulates the
lengt h of the result.
var c ByteCounter
c.Write([]byte("hello"))
fmt.Println(c) // "5", = len("hello")
c=0//reset the counter
var name = "Dolly"
fmt.Fprintf(&c, "hello, %s", name)
fmt.Println(c) // "12", = len("hello, Dolly")
Besides io.Writer,there isanother interface ofgre atimp ortance tothe fmt package.
Fprintf and Fprintln prov ide a way for typ es to control how their values areprint ed. In
Section2.5, wedefine d a String method for the Celsius type sothattemperatures wou ld
pr int as "100°C",and inSec tion 6.5 weequip ped *IntSet with a String method sothatsets
wouldberendered usingtradition alset not ation like "{1 2 3}".Declaring a String method
makesatyp e satisf y on e of the most widely usedint erfaces ofall, fmt.Stringer:
package fmt
// The String method is used to print values passed
// as an operand to any format that accepts a string
// or to an unformatted printer such as Print.
type Stringer interface {
String() string
}
Well explain how the fmt packagedis cov ers whichvalues sat isf y this int erface in Sec tion 7.10.
Exercis e 7.1: Usingthe ide as from ByteCounter,imp lementcount ers for words and for lines.
Yo u wi l l nd bufio.ScanWords us eful.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
174 CHAPTER 7. INTERFACES
Exercis e 7.2: Wr ite a function CountingWriter with thesig naturebelow that, given an
io.Writer,retur nsanew Writer that wraps the original, and a point ertoan int64 var iable
that at anymom ent containsthe numberofbytes writt entothe new Writer.
func CountingWriter(w io.Writer) (io.Writer, *int64)
Exercis e 7.3: Wr ite a String method for the *tree type in gopl.io/ch4/treesort (§4.4)
that reveals the sequence ofvalues in the tre e.
7.2. Int erfaceTypes
An int erface typ e sp ecifies a set ofmet hodsthatacon crete typ e mu stpossess tobecon sidered
an inst anceofthatint erface.
The io.Writer type isone ofthe most widely usedint erfaces because itprovides an abstrac-
tion of all the typ es to whichbytes can bewritt en, whichincludes files, memor y buffers, net-
work conne ction s,HTTPclients, archivers, hashers,and soon. The io packagedefinesmany
ot her usefulint erfaces. A Reader repres ents anytyp e from whichyou can readbytes, anda
Closer is any value thatyou can clos e,suchasafile oranet wor k connec tion.(By now youve
prob ably not ice d thenamingconvent ion for manyofGossingle-met hod int erfaces.)
package io
type Reader interface {
Read(p []byte) (n int, err error)
}
type Closer interface {
Close() error
}
Looking far ther, wefind declarat ions ofnew int erface typ es as combinat ions ofexist ing ones.
Here are two examples:
type ReadWriter interface {
Reader
Writer
}
type ReadWriteCloser interface {
Reader
Writer
Closer
}
Thesyntaxusedabove , whichres embles str uct emb edding, lets usnameanother interface as a
shorthand for writing out all ofits met hods. Thisiscal le d embedding an interface.Wecou ld
have writt en io.ReadWriter withoutemb edding, alb eit lesssuccinctly,likethis:
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 7.3. INTERFACE SATISFACTION 175
type ReadWriter interface {
Read(p []byte) (n int, err error)
Write(p []byte) (n int, err error)
}
or evenusingamixtureofthe two sty les:
type ReadWriter interface {
Read(p []byte) (n int, err error)
Writer
}
Al l thre e de clarat ions havethe sameeffec t.The order in whichthe methodsapp ear isimma-
terial.All thatmatters isthe set ofmet hods.
Exercis e 7.4: The strings.NewReader func tionretur nsavalue thatsat isfies the io.Reader
interface (andothers) byreadingfro m itsargument, a str ing . Implementasimpleversionof
NewReader yourself,and use ittomakethe HTMLparser(§5.2) takeinp utfro m astr ing .
Exercis e 7.5: The LimitReader func tioninthe io packageaccepts an io.Reader r anda
numb erofbytes n,and retur nsanother Reader that reads fro m r butrep orts an end-of-file
condit ion after n bytes. Imp lementit.
func LimitReader(r io.Reader, n int64) io.Reader
7.3. Int erfaceSatisfaction
Atyp e sati ses an interface if itpossess esall the methodsthe int erface requires. For example,
an *os.File satisfies io.Reader, Writer, Closer,and ReadWriter.A*bytes.Buffer sat-
isfies Reader, Writer,and ReadWriter,but doesnot sat isf y Closer becaus e it doesnot havea
Close method.Asashorthand,Goprogrammersoften say thatacon crete typ e ‘‘is a’’ partic u-
larint erface typ e,meaningthatitsat isfies the int erface.For example, a *bytes.Buffer is an
io.Writer;an *os.File is an io.ReadWriter.
Theassig nabilit y rule (§2.4.2) for int erfaces isver y simple: an expressionmay beassig ned to
an interface only if its typ e satisfies the int erface.So:
var w io.Writer
w=os.Stdout // OK: *os.File has Write method
w=new(bytes.Buffer) // OK: *bytes.Buffer has Write method
w=time.Second // compile error: time.Duration lacks Write method
var rwc io.ReadWriteCloser
rwc = os.Stdout // OK: *os.File has Read, Write, Close methods
rwc = new(bytes.Buffer) // compile error: *bytes.Buffer lacks Close method
Thisruleapp lies evenwhenthe rig ht-handside isits elf an interface:
w=rwc // OK: io.ReadWriteCloser has Write method
rwc = w // compile error: io.Writer lacks Close method
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
176 CHAPTER 7. INTERFACES
Becaus e ReadWriter and ReadWriteCloser include allthe methodsof Writer,any typ e that
satisfies ReadWriter or ReadWriteCloser ne cessarily sat isfies Writer.
Before wegofur ther, weshouldexplain one subtlet y in whatitmeans for a typ e to havea
method.Recal l from Sec tion 6.2 thatfor eachnamed con crete typ e T,som e of itsmet hods
have a receiveroftyp e T itself where asothersrequirea*T pointer. Recal l also thatitislegal to
call a *T method onanargumentoftyp e T so lon g as the argumentisavariab le;the compi ler
implicitlytakes its address. But thisismeresyntactic sugar : avalue oftyp e T do es notpossess
al l themet hodsthata*T pointerdoes, and as a resultitmig htsat isf y fe wer interfaces.
An examplewill makethiscle ar.The String method ofthe IntSet type fro m Section6.5
re quires a point erreceiver, sowecannot cal l that met hod onanon-addressable IntSet value:
type IntSet struct { /* ... */ }
func (*IntSet) String() string
var _ = IntSet{}.String() // compile error: String requires *IntSet receiver
butwecan cal l it on an IntSet var iable:
var s IntSet
var _ = s.String() // OK: s is a variable and &s has a String method
Ho wever,since only *IntSet hasaString method,only *IntSet satisfies the fmt.Stringer
interface:
var _ fmt.Stringer = &s // OK
var _ fmt.Stringer = s // compile error: IntSet lacks String method
Section12.8 includes a program thatprintsthe methodsofanarbit rar y value,and the
godoc -analysis=type to ol(§10.7.4) displays the methodsofeachtyp e andthe rel ation ship
betweenint erfaces andcon crete typ es.
Like anenv elope thatwraps andcon ceals the letteritholds,anint erface wraps andcon ceals
thecon crete typ e andvalue thatitholds.Onlythe methodsreveale d by the int erface typ e may
be cal le d,even if the con crete typ e hasothers:
os.Stdout.Write([]byte("hello")) // OK: *os.File has Write method
os.Stdout.Close() // OK: *os.File has Close method
var w io.Writer
w=os.Stdout
w.Write([]byte("hello")) // OK: io.Writer has Write method
w.Close() // compile error: io.Writer lacks Close method
An int erface wit h more met hods, suchas io.ReadWriter,tel lsusmoreabout the values it
cont ains, andplaces gre aterdemands onthe typ es that implementit, thandoesanint erface
with fewer met hodssuchas io.Reader.Sowhatdoesthe typ e interface{},whichhas no
methods at all,tel l us about the con crete typ es that sat isf y it?
Thatsrig ht: not hing. Thismay seemuseless, but infac t thetyp e interface{},whichis
called the emptyint erface type,isindispens able. Because the emp tyint erface typ e pl aces no
demands onthe typ es that sat isf y it,wecan assig n any value tothe emp tyint erface.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 7.3. INTERFACE SATISFACTION 177
var any interface{}
any = true
any = 12.34
any = "hello"
any = map[string]int{"one": 1}
any = new(bytes.Buffer)
Although itwasntobv iou s,weve beenusingthe emp tyint erface typ e since the ver y rs t
exampleinthisbook, because itiswhatallowsfunctions like fmt.Println,or errorf in
Section5.7, toacceptarguments ofany typ e.
Of course,hav ingcre ate d an interface{} value cont ainingaboole an, oat, str ing , map,
pointer, orany other typ e,wecan do not hingdirec tly tothe value itholds since the int erface
hasnomet hods. Weneed a way toget the value backout again. Well see how todothat
usingaty peass ertion in Sec tion 7.10.
Sinceint erface sat isfac tiondep ends onlyonthe methodsofthe two typ es invo l ved, there isno
ne e d to declare the rel ation shipbet weena con crete typ e andthe int erfaces itsat isfies. That
said,itisocc asionallyusefultodocumentand ass ert the rel ation shipwhenitisint ended but
notother wis e enforce d by the program. Thedeclarat ionbelow ass ertsatcompi letimethata
value oftyp e *bytes.Buffer satisfies io.Writer:
// *bytes.Buffer must satisfy io.Writer
var w io.Writer = new(bytes.Buffer)
We needntallocateanew var iable since anyvalue oftyp e *bytes.Buffer wi l l do,even nil,
whichwewrite as (*bytes.Buffer)(nil) usinganexplicitconversion. And since wenever
intend torefer to w,wecan replace itwit h theblank identifier.Toget her,these changesgiveus
this morefrugal variant:
// *bytes.Buffer must satisfy io.Writer
var _ io.Writer = (*bytes.Buffer)(nil)
No n-empt y interface typ es such as io.Writer aremostoften sat isfied byapoint ertyp e,par-
ticularlywhenone ormoreofthe int erface met hodsimp lies som e kind ofmut ation tothe
re ceiver, asthe Write method does. A pointertoastr uct isanesp eci ally commonmet hod-
bear ingtyp e.
Butpoint ertyp es arebynomeans the onlytyp es that sat isf y interfaces, andevenint erfaces
with mut atormet hodsmay besat isfied byone ofGosother reference typ es. Weve seenexam-
ples ofslice typ es with met hods(geometry.Path6.1) andmap typ es with met hods
(url.Values6.2.1), andlater we ll see a functiontyp e with met hods(http.HandlerFunc,
§7.7). Evenbasic typ es maysat isf y interfaces; as wesaw in Sec tion 7.4, time.Duration sat-
isfies fmt.Stringer.
Acon crete typ e maysat isf y many unrel ate d interfaces. Con sider a program thatorganizes or
sellsdig itize d cult ural artifac ts li kemusic, films, andbooks. Itmig htdefine the fol low ing set
of con crete typ es:
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
178 CHAPTER 7. INTERFACES
Album
Book
Movie
Magazine
Podcast
TVEpisode
Track
We can express eachabstrac tionofint erest asanint erface.Som e prop erties arecommontoall
ar tifac ts, such asatit le, a cre ation date, and a listofcre ators (author s or artists).
type Artifact interface {
Title() string
Creators() []string
Created() time.Time
}
Ot her pro per ties arerestr icted tocer tain typ es of artifac ts. Prop erties ofthe print edwordare
re levantonlytobooks andmagazines, where asonlymov ies andTVepi sodes haveascreen
resolut ion.
type Text interface {
Pages() int
Words() int
PageSize() int
}
type Audio interface {
Stream() (io.ReadCloser, error)
RunningTime() time.Duration
Format() string // e.g., "MP3", "WAV"
}
type Video interface {
Stream() (io.ReadCloser, error)
RunningTime() time.Duration
Format() string // e.g., "MP4", "WMV"
Resolution() (x, y int)
}
Thes e interfaces arebut oneusefulway togro uprel ate d concrete typ es together andexpress
thefacets the y sh are incommon. Wemay discov erother gro upingslater.For example, ifwe
nd weneed tohandle Audio and Video it ems inthe sameway,wecan define a Streamer
interface torepresent their common asp ectswit houtchang ing any exist ing typ e de clarat ions.
type Streamer interface {
Stream() (io.ReadCloser, error)
RunningTime() time.Duration
Format() string
}
Each gro uping ofcon crete typ es basedontheir share d behavior s canbeexpress edasanint er-
face typ e.Unlikeclass-b ased langu ages, in whichthe set ofint erfaces sat isfied byaclass is
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 7.4. PARSING FLAGS WITH FLAG.VALUE179
explicit, in Gowecan define newabstrac tions orgro upingsofint erest whenweneed them,
withoutmodif yingthe declarat ionofthe con crete typ e.Thisispar tic ularlyusefulwhenthe
concrete typ e comesfro m apackagewritt enbyadif ferentaut hor.Ofcours e, theredoneed to
be underly ing commonalities in the con crete typ es.
7.4. Parsing Flags with flag.Value
In thissec tion,well see how another stand ard int erface, flag.Value,helps usdefine new
notation s forcommand-line flags. Con sider the program below,whichsle eps foraspecified
period oftime.
gopl.io/ch7/sleep
var period = flag.Duration("period", 1*time.Second, "sleep period")
func main() {
flag.Parse()
fmt.Printf("Sleeping for %v...", *period)
time.Sleep(*period)
fmt.Println()
}
Before itgoestosle ep it prints the timeper iod. The fmt packagecal lsthe time.Durations
String method toprint the per iodnot asanumberofnanos econd s,but inauser-friendly
notation:
$gobuild gopl.io/ch7/sleep
$./sleep
Sleeping for 1s...
By defau lt, the sleepper iodisone secon d,but it can becontrol led through the -period com-
mand-line flag . The flag.Duration func tioncre atesaflag variableoftyp e time.Duration
andallowsthe usertospecif y thedurat ioninavar ietyofuser-friendlyfor mats, includingthe
same notation printe d by the String method.Thissymmetr y of desig n le adstoanice user
interface.
$./sleep -period 50ms
Sleeping for 50ms...
$./sleep -period 2m30s
Sleeping for 2m30s...
$./sleep -period 1.5h
Sleeping for 1h30m0s...
$./sleep -period "1 day"
invalid value "1 day" for flag -period: time: invalid duration 1 day
Becaus e durat ion-value d flags aresouseful, thisfeature isbui ltint o the flag package, but its
easy todefine newflag not ation s forour own data typ es. We need onlydefine a typ e that sat-
isfies the flag.Value interface,whose declarat ionisbelow :
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
180 CHAPTER 7. INTERFACES
package flag
// Value is the interface to the value stored in a flag.
type Value interface {
String() string
Set(string) error
}
The String method for mats the flagsvalue for use incommand-linehelpmessages; thu s
ever y flag.Value is als o a fmt.Stringer.The Set method parsesits str ing argumentand
up dates the flag value.Ineffec t,the Set method isthe inv ers e of the String method,and itis
go o d prac tice for themtouse the samenot ation.
Letsdefine a celsiusFlag type thatallowsatemperaturetobespecified inCel siu s,orin
Fahren heitwit h an appropriateconversion. Not ice that celsiusFlag embeds a Celsius
(§2.5), there bygetting a String method for fre e.Tosat isf y flag.Value,weneed onlydeclare
the Set method:
gopl.io/ch7/tempconv
// *celsiusFlag satisfies the flag.Value interface.
type celsiusFlag struct{ Celsius }
func (f *celsiusFlag) Set(s string) error {
var unit string
var value float64
fmt.Sscanf(s, "%f%s", &value, &unit) // no error check needed
switch unit {
case "C", "°C":
f.Celsius = Celsius(value)
return nil
case "F", "°F":
f.Celsius = FToC(Fahrenheit(value))
return nil
}
return fmt.Errorf("invalid temperature %q", s)
}
Thecal l to fmt.Sscanf pars esaoating-p ointnumber(value)and a str ing (unit)fro m the
input s.Alt hough one mustusu allyche ck Sscanfserror result, in thiscas e we dontneed to
becaus e if there was a pro blem, noswitch cas e wi l l match.
The CelsiusFlag func tionbelow wraps itall up. Tothe cal ler,itretur nsapoint ertothe Cel-
sius eldemb edde d within the celsiusFlag var iable f.The Celsius eldisthe var iable
that will beupdated bythe Set method dur ing flags pro cessing. The cal l to Var addsthe flag
to the app lic ationsset ofcommand-line flags, the globalvar iable flag.CommandLine.
Prog ramswit h unusuallycomplex command-lineint erfaces may haveseveral variables ofthis
type.The cal l to Var assig nsa*celsiusFlag argumenttoaflag.Value parameter,causing
thecompi ler toche ckthat *celsiusFlag hasthe necessary met hods.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 7.5. INTERFACE VALUES 181
// CelsiusFlag defines a Celsius flag with the specified name,
// default value, and usage, and returns the address of the flag variable.
// The flag argument must have a quantity and a unit, e.g., "100C".
func CelsiusFlag(name string, value Celsius, usage string) *Celsius {
f:=celsiusFlag{value}
flag.CommandLine.Var(&f, name, usage)
return &f.Celsius
}
No w we can start usingthe new flag in our programs:
gopl.io/ch7/tempflag
var temp = tempconv.CelsiusFlag("temp", 20.0, "the temperature")
func main() {
flag.Parse()
fmt.Println(*temp)
}
Heresatypic al session:
$gobuild gopl.io/ch7/tempflag
$./tempflag
20°C
$./tempflag -temp -18C
-18°C
$./tempflag -temp 212°F
100°C
$./tempflag -temp 273.15K
invalid value "273.15K" for flag -temp: invalid temperature "273.15K"
Usage of ./tempflag:
-temp value
the temperature (default 20°C)
$./tempflag -help
Usage of ./tempflag:
-temp value
the temperature (default 20°C)
Exercis e 7.6: Addsup por t forKelvintemperatures to tempflag.
Exercis e 7.7: Expl ain why the helpmessage contains °C when the defau ltvalue of 20.0 do es
not.
7.5. Int erfaceValues
Conceptu ally, a value ofanint erface typ e,or interface value,has two components, a concrete
type and a value ofthattyp e.These are cal le d theint erfaces dy namic type and dy namic valu e.
Fo r astaticallytyp edlangu agelikeGo, typ es areacompi le-t imecon cept, soatyp e is not a
value.Inour conceptu almodel,aset ofvalues cal le d ty pedes cript o rs prov ide infor mat ion
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
182 CHAPTER 7. INTERFACES
ab out eachtyp e,suchasits nameand methods. Inanint erface value,the typ e comp onent is
repres ente d by the appro priatetyp e des crip tor.
In the fourstatementsbelow,the var iable w takesonthree dif ferentvalues. (Theinitial and
nalvalues arethe same.)
var w io.Writer
w=os.Stdout
w=new(bytes.Buffer)
w=nil
Letstakeaclos er lo okatthe value anddynamic beh avior of w af ter eachstatement.The rs t
st atement declares w:
var w io.Writer
In Go, var iables arealways initialize d to a wel l-define d value,and int erfaces arenoexception.
Thezerovalue for anint erface has bot h itstyp e andvalue comp onentsset to nil (Figure7.1).
Figure 7.1. Anil int erface value.
An int erface value isdes crib edasnil ornon-ni l basedonits dynamic typ e,sothisisanil
interface value.You can testwhether an interface value isnil using w==nil or w!=nil.
Callingany met hod ofanil int erface value causesapanic:
w.Write([]byte("hello")) // panic: nil pointer dereference
Thesecon d st atement assig nsavalue oftyp e *os.File to w:
w=os.Stdout
Thisassig nmentinv olves an implicitconversionfro m acon crete typ e to anint erface typ e,and
is equivalenttothe explicitconversion io.Writer(os.Stdout).Aconv ersionofthiskind,
whet her explicitorimp licit, capturesthe typ e andthe value ofits operand. The int erface
valuesdynamic typ e is set tothe typ e des crip tor for the point ertyp e *os.File,and its
dy namic value holds a copyof os.Stdout,whichisapointertothe os.File var iable rep-
resent ing the stand ard out put of the pro cess(Figure7.2).
Figure 7.2. An int erface value cont ainingan *os.File pointer.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 7.5. INTERFACE VALUES 183
Callingthe Write method onanint erface value cont ainingan *os.File pointercausesthe
(*os.File).Write method tobecal le d.The cal l pr ints "hello".
w.Write([]byte("hello")) // "hello"
In general,wecannot know at compi letimewhatthe dynamic typ e of anint erface value will
be,soa cal l thro ugh anint erface mustuse dy namic dispatch.Ins teadofadirec t call,the com-
pi ler mustgeneratecodetoobt ain the addressofthe method named Write from the typ e
des crip tor,thenmakeanindirec t call tothataddress. Thereceiverargumentfor the cal l is a
copy ofthe int erfacesdynamic value, os.Stdout.The effec t is asifwehad made thiscal l
direc tly :
os.Stdout.Write([]byte("hello")) // "hello"
Thethirdstatement assig nsavalue oftyp e *bytes.Buffer to the int erface value:
w=new(bytes.Buffer)
Thedynamic typ e is now *bytes.Buffer andthe dynamic value isapoint ertothe newly
al locate d buffer (Figure7.3).
Figure 7.3. An int erface value cont aininga*bytes.Buffer pointer.
Acal l to the Write method usesthe samemechanism as before:
w.Write([]byte("hello")) // writes "hello" to the bytes.Buffer
Thistime, the typ e des crip tor is *bytes.Buffer,sothe (*bytes.Buffer).Write method is
called,wit h theaddressofthe buf fer as the value ofthe receiverparameter.The cal l append s
"hello" to the buf fer.
Final ly, the fourthstatement assig ns nil to the int erface value:
w=nil
Thisres ets both its comp onentsto nil,restoring w to the samestate aswhenitwas decl are d,
whichwas shown inFigure7.1.
An int erface value can holdarbit rar ily large dynamic values. For example, the time.Time
type,whichrepresentsanins tantintime, isastr uct typ e with several unexp orted elds.Ifwe
create anint erface value fro m it,
var x interface{} = time.Now()
theresultmig htlooklikeFigure7.4. Con ceptu ally, the dynamic value always fits inside the
interface value,nomatterhow large its typ e.(Thisisonlyacon ceptu almodel;arealist ic
implementation isquite dif ferent.)
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
184 CHAPTER 7. INTERFACES
Figure 7.4. An int erface value holdingatime.Time st ruc t.
Interface values may becompare d using == and !=.Two int erface values areequ alifbot h are
ni l,oriftheir dynamic typ es areidenticaland their dynamic values areequ alaccordingtothe
usualbeh avior of == forthattyp e.Because int erface values arecomparable,the y maybeused
as the keysofamap or as the operandofaswitch statement.
Ho wever,iftwo int erface values arecompare d andhavethe samedynamic typ e,but thattyp e
is not comparable (a slice,for ins tance), thenthe comparisonfai lswit h apanic:
var x interface{} = []int{1, 2, 3}
fmt.Println(x == x) // panic: comparing uncomparable type []int
In thisrespect,int erface typ es areunu sual. Other typ es areeit her safelycomparable (like
basic typ es andpoint ers)ornot comparable at all(li keslices, maps, andfunctions), but when
comp aring int erface values oraggregatetyp es that cont ain interface values, wemustbeaware
of the pot ent ial for a panic. A simi lar riskexistswhenusingint erfaces as map keysorswitch
op erands.Onlycompare int erface values if you are cer tain thatthe y cont ain dynamic values
of comparable typ es.
Wh enhandlingerror s,ordur ingdebug ging, itisoften helpf ultorep ort the dynamic typ e of
an interface value.For that, weuse the fmt packages %T verb:
var w io.Writer
fmt.Printf("%T\n", w) // "<nil>"
w=os.Stdout
fmt.Printf("%T\n", w) // "*os.File"
w=new(bytes.Buffer)
fmt.Printf("%T\n", w) // "*bytes.Buffer"
Internal ly, fmt us esreflec tion toobt ain the nameofthe int erfacesdynamic typ e.Well lookat
reec tion inChapt er12.
7.5.1. Caveat: An Int erfaceContaining a Nil PointerIsNon-Nil
Anil int erface value,whichcontainsnovalue at all, isnot the sameasanint erface value con-
tainingapoint erthathappens tobenil.Thissubtledistinc tioncre atesatrap into whichevery
Go programmer has stumb led.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 7.5. INTERFACE VALUES 185
Consider the program below.Wit h debug setto true,the main functioncol lec tsthe out put of
thefunction f in a bytes.Buffer.
const debug = true
func main() {
var buf *bytes.Buffer
if debug {
buf = new(bytes.Buffer) // enable collection of output
}
f(buf) // NOTE: subtly incorrect!
if debug {
// ...use buf...
}
}
// If out is non-nil, output will be written to it.
func f(out io.Writer) {
// ...do something...
if out != nil {
out.Write([]byte("done!\n"))
}
}
We mig htexp ect thatchang ing debug to false woulddis ablethe col lec tion of the out put,but
in factitcausesthe program topanic dur ingthe out.Write call:
if out != nil {
out.Write([]byte("done!\n")) // panic: nil pointer dereference
}
Wh en main calls f,itassig nsanil point eroftyp e *bytes.Buffer to the out parameter,sothe
dy namic value of out is nil.How ever, its dynamic typ e is *bytes.Buffer,meaningthat out
is a non-ni l interface cont aininganil point ervalue (Figure7.5), sothe defensive che ck
out != nil is still true.
Figure 7.5. Anon-ni l interface cont aininganil point er.
As before, the dynamic disp atch mechanism deter mines that (*bytes.Buffer).Write mu st
be cal le d butthistimewit h areceivervalue thatisnil.For som e types, suchas *os.File, nil
is a valid receiver(§6.2.1), but *bytes.Buffer is not among them. Themet hod iscal le d,but
it panics as ittries toaccessthe buf fer.
Thepro blem isthatalt hough a nil *bytes.Buffer pointerhas the methodsneeded tosat isf y
theint erface,itdoesntsat isf y the behavioral re quirements ofthe int erface.Inpar tic ular, the
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
186 CHAPTER 7. INTERFACES
call violatesthe imp licitpre con dit ion of (*bytes.Buffer).Write that its receiverisnot nil,
so assig ningthe nil point ertothe int erface was a mist ake . Thesolut ion istochange the typ e
of buf in main to io.Writer,there byavoidingthe assig nmentofthe dysf unctionalvalue to
theint erface in the rs t pl ace:
var buf io.Writer
if debug {
buf = new(bytes.Buffer) // enable collection of output
}
f(buf) // OK
No w that weve cov ere d themechanics ofint erface values, letstakealook at som e more
importantint erfaces fro m Gosstand ard librar y.Inthe next three sec tion s,well see how int er-
faces areusedfor sor ting, web ser ving, and error handling.
7.6. Sorting with sort.Interface
Like str ing for matting , sortingisafre quentlyusedoperat ioninmanyprograms. Alt hough a
minimal Quicks ort can bewritt eninabout 15lines, a robustimp lementation ismuchlon g er,
anditisnot the kindofcodeweshouldwishtowrite ane w or copyeachtimeweneed it.
Fo rtunate ly, the sort packageprovides in-place sor tingofany sequence accordingtoany
order ingfunction. Its designisrat her unusual. Inmanylangu ages, the sor tingalgor it hmis
asso ciate d with thesequence dat a type,whi le theorder ingfunctionisass oci ated wit h thetyp e
of the elements. Bycontrast, Gos sort.Sort func tionassumesnot hingabout the represen-
tation of eit her the sequence orits elements. Ins tead, itusesanint erface, sort.Interface,to
sp ecif y thecontrac t betweenthe gener ic sort algor it hmand eachsequence typ e that may be
sorted.Animp lementation of thisint erface deter mines bot h thecon crete represent ation of
thesequence,whichisoften a slice,and the desired order ingofits elements.
An in-place sor t algor it hmneedsthree thingsthelengt h of the sequence,ameans ofcom-
paring two elements, andaway toswap two elementsso the y arethe three met hodsof
sort.Interface:
package sort
type Interface interface {
Len() int
Less(i, j int) bool // i, j are indices of sequence elements
Swap(i, j int)
}
To sor t anysequence,weneed todefine a typ e that implements these three met hods, then
apply sort.Sort to anins tance ofthattyp e.Asperhaps the simplestexample, con sider
sortingaslice ofstr ings. Thenew typ e StringSlice andits Len, Less,and Swap methodsare
shown below.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 7.6. SORTING WITH SORT.INTERFACE 187
type StringSlice []string
func (p StringSlice) Len() int {return len(p) }
func (p StringSlice) Less(i, j int) bool { return p[i] < p[j] }
func (p StringSlice) Swap(i, j int) {p[i], p[j] = p[j], p[i] }
No w we can sor t aslice ofstr ings, names,byconvertingthe slice toaStringSlice li kethis:
sort.Sort(StringSlice(names))
Theconversionyieldsaslice value wit h thesamelengt h, capacity,and underly ing array as
names butwit h atyp e that has the three met hodsrequired for sor ting.
Sortingaslice ofstr ingsissocommonthatthe sort packageprovides the StringSlice type,
as wel l as a functioncal le d Strings so thatthe cal l ab ove can besimplified to
sort.Strings(names).
Thetechnique hereiseasi lyadapt edtoother sor t orders, for ins tance,toignorecapit alizat ion
or speci al ch arac ters. (TheGoprogram thatsor tsindex ter msand pagenumbers for this
bookdoesthis, wit h ext ra log ic forRom annumerals.) For morecomplic ated sor ting, weuse
thesameide a, butwit h more complic ated dat a st ruc tures ormorecomplic ated imp lemen-
tation s of the sort.Interface methods.
Ourrunningexamplefor sor tingwill beamusic playlist, displaye d as a table.Eachtrackisa
singlerow,and eachcolumn isanatt ribut e of thattrack, like artist, tit le, and runningtime.
Im agine thatagraphic al us erint erface presentsthe table,and thatclicking the head ofacol-
umn causesthe playlisttobesor ted bythatatt ribut e;clicking the samecolumn headagain
re versesthe order.Letslook at whatmig hthappen in respons e to eachclick.
Thevar iable tracks belowcontainsaplaylist. (Oneofthe aut hor s ap olog izes forthe other
authorsmusic al tastes.) Eachelementisindirec t,apoint ertoaTrack.Alt hough the code
belowwou ldwor k if westore d the Tracksdirec tly,the sor t func tionwill swap manypairsof
elements, soitwill run fasterifeachelementisapoint er, whichisasinglemachineword,
instead of an ent ire Track,whichmig htbeeig htwords ormore.
gopl.io/ch7/sorting
type Track struct {
Title string
Artist string
Album string
Year int
Length time.Duration
}
var tracks = []*Track{
{"Go", "Delilah", "From the Roots Up", 2012, length("3m38s")},
{"Go", "Moby", "Moby", 1992, length("3m37s")},
{"Go Ahead", "Alicia Keys", "As I Am", 2007, length("4m36s")},
{"Ready 2 Go", "Martin Solveig", "Smash", 2011, length("4m24s")},
}
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
188 CHAPTER 7. INTERFACES
func length(s string) time.Duration {
d, err := time.ParseDuration(s)
if err != nil {
panic(s)
}
return d
}
The printTracks func tionprintsthe playlist as a table.Agraphic al displaywou ldbenicer,
butthislit tle routine usesthe text/tabwriter packagetopro duce a table whose columnsare
ne atlyalig ned and padde d as shown below.Obs erve that *tabwriter.Writer satisfies
io.Writer.Itcol lec tseachpie ce of dat a wr itt entoit; its Flush method for mats the ent ire ta-
bleand writesitto os.Stdout.
func printTracks(tracks []*Track) {
const format = "%v\t%v\t%v\t%v\t%v\t\n"
tw := new(tabwriter.Writer).Init(os.Stdout, 0, 8, 2, '',0)
fmt.Fprintf(tw, format, "Title", "Artist", "Album", "Year", "Length")
fmt.Fprintf(tw, format, "-----", "------", "-----", "----", "------")
for _, t := range tracks {
fmt.Fprintf(tw, format, t.Title, t.Artist, t.Album, t.Year, t.Length)
}
tw.Flush() // calculate column widths and print table
}
To sor t theplaylistbythe Artist eld, wedefine a newslice typ e with thenecessary Len,
Less,and Swap methods, analogou s to whatwedid for StringSlice.
type byArtist []*Track
func (x byArtist) Len() int {return len(x) }
func (x byArtist) Less(i, j int) bool { return x[i].Artist < x[j].Artist }
func (x byArtist) Swap(i, j int) {x[i], x[j] = x[j], x[i] }
To cal l thegener ic sort routine,wemustfirs t conv ert tracks to the newtyp e, byArtist,that
definesthe order :
sort.Sort(byArtist(tracks))
Af ter sor tingthe slice byartist, the out put fro m printTracks is
Title Artist Album Year Length
----- ------ ----- ---- ------
Go Ahead Alicia Keys As I Am 2007 4m36s
Go Delilah From the Roots Up 2012 3m38s
Ready 2 Go Martin Solveig Smash 2011 4m24s
Go Moby Moby 1992 3m37s
If the userrequests ‘‘sort byartist’’ asecon d time,well sor t thetracks in reverse.Weneednt
define a newtyp e byReverseArtist with aninv erted Less method,how ever, since the sort
packageprovides a Reverse func tionthattransfor msany sor t order toits invers e.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 7.6. SORTING WITH SORT.INTERFACE 189
sort.Sort(sort.Reverse(byArtist(tracks)))
Af ter reverse-s ortingthe slice byartist, the out put fro m printTracks is
Title Artist Album Year Length
----- ------ ----- ---- ------
Go Moby Moby 1992 3m37s
Ready 2 Go Martin Solveig Smash 2011 4m24s
Go Delilah From the Roots Up 2012 3m38s
Go Ahead Alicia Keys As I Am 2007 4m36s
The sort.Reverse func tiondes ervesaclos er lo oksince itusescomposition (§6.3), whichis
an importantide a. The sort packagedefinesanunexp orted typ e reverse,whichisastr uct
that embeds a sort.Interface.The Less method for reverse callsthe Less method ofthe
embedde d sort.Interface value,but wit h theindices ip ped,reversingthe order ofthe sor t
resu lts.
package sort
type reverse struct{ Interface } // that is, sort.Interface
func (r reverse) Less(i, j int) bool { return r.Interface.Less(j, i) }
func Reverse(data Interface) Interface { return reverse{data} }
Len and Swap,the other two met hodsof reverse,are imp licitlyprovide d by the original
sort.Interface value because itisanemb edde d eld. The exp orted function Reverse
returns an ins tance ofthe reverse type thatcontainsthe original sort.Interface value.
To s or t by a dif ferentcolumn, wemustdefine a newtyp e,suchas byYear:
type byYear []*Track
func (x byYear) Len() int {return len(x) }
func (x byYear) Less(i, j int) bool { return x[i].Year < x[j].Year }
func (x byYear) Swap(i, j int) {x[i], x[j] = x[j], x[i] }
Af ter sor ting tracks by yearusing sort.Sort(byYear(tracks)), printTracks shows a
chro nolog ical list ing:
Title Artist Album Year Length
----- ------ ----- ---- ------
Go Moby Moby 1992 3m37s
Go Ahead Alicia Keys As I Am 2007 4m36s
Ready 2 Go Martin Solveig Smash 2011 4m24s
Go Delilah From the Roots Up 2012 3m38s
Fo r ever y slice elementtyp e andevery order ingfunctionweneed,wedeclare a new imp le-
ment ation of sort.Interface.Asyou can see,the Len and Swap methodshaveidenticaldef-
inition s forall slice typ es. In the next example, the con crete typ e customSort combines a slice
with a func tion, letting usdefine a newsor t order bywriting onlythe comparisonfunction.
In cidentally, the con crete typ es that implement sort.Interface arenot always slices; cus-
tomSort is a str uct typ e.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
190 CHAPTER 7. INTERFACES
type customSort struct {
t[]*Track
less func(x, y *Track) bool
}
func (x customSort) Len() int {return len(x.t) }
func (x customSort) Less(i, j int) bool { return x.less(x.t[i], x.t[j]) }
func (x customSort) Swap(i, j int) {x.t[i], x.t[j] = x.t[j], x.t[i] }
Letsdefine a multi-t ier order ingfunctionwhose primary sor t ke y is the Title,whose
second ary key isthe Year,and whose ter tiary key isthe runningtime, Length.Heresthe cal l
to Sort using an anony mou s order ingfunction:
sort.Sort(customSort{tracks, func(x, y *Track) bool {
if x.Title != y.Title {
return x.Title < y.Title
}
if x.Year != y.Year {
return x.Year < y.Year
}
if x.Length != y.Length {
return x.Length < y.Length
}
return false
}})
Andheresthe result. Not ice thatthe tie bet weenthe two tracks tit led ‘‘Go’’ is bro ken in favor
of the older one.
Title Artist Album Year Length
----- ------ ----- ---- ------
Go Moby Moby 1992 3m37s
Go Delilah From the Roots Up 2012 3m38s
Go Ahead Alicia Keys As I Am 2007 4m36s
Ready 2 Go Martin Solveig Smash 2011 4m24s
Although sor tingasequence oflengt h n re quires O(n log n)comparisonoperat ions,testing
whet her a sequence isalready sor ted requires at most n1comparisons.The IsSorted func-
tion fro m the sort packageche cks thisfor us. Like sort.Sort,itabstrac ts both thesequence
andits order ingfunctionusing sort.Interface,but it never cal lsthe Swap method:This
co de demon strates the IntsAreSorted and Ints func tions and the IntSlice type:
values := []int{3, 1, 4, 1}
fmt.Println(sort.IntsAreSorted(values)) // "false"
sort.Ints(values)
fmt.Println(values) // "[1 1 3 4]"
fmt.Println(sort.IntsAreSorted(values)) // "true"
sort.Sort(sort.Reverse(sort.IntSlice(values)))
fmt.Println(values) // "[4 3 1 1]"
fmt.Println(sort.IntsAreSorted(values)) // "false"
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 7.7. THE HTTP.HANDLER INTERFACE 191
Fo r conv enience,the sort packageprovides versions ofits functions and typ es sp eci alizedfor
[]int, []string,and []float64 usingtheir natural order ings. For other typ es, such as
[]int64 or []uint,were onour own,thoug h thepat h is short.
Exercis e 7.8: Many GUIsprovide a table widget wit h astatefulmulti-t ier sort:the primary
sort key isthe most recentlyclicke d column head, the secon dar y sort key isthe secon d-most
re centlyclicke d column head, and soon. Define animp lementation of sort.Interface for
us e by suchatable.Compare thatappro ach wit h repeated sor tingusing sort.Stable.
Exercis e 7.9: Us e the html/template package(§4.6) toreplace printTracks with a func tion
that displays the tracks as an HTML table.Use the solut ion tothe pre vious exercis e to arrange
that eachclickonacolumn headmakes an HTTP requesttosor t thetable.
Exercis e 7.10: The sort.Interface type can beadapt edtoother uses. Write a function
IsPalindrome(s sort.Interface) bool that rep ortswhether the sequence s is a palin-
drom e,inother words,reversingthe sequence wou ldnot change it. Assume thatthe elements
at indices i and j areequ alif !s.Less(i, j) && !s.Less(j, i).
7.7. The http.Handler Int erface
In Chapt er1,wesaw a glimps e of how touse the net/http packagetoimp lementweb clients
(§1.5) andser vers(§1.7). Inthissec tion,well lookmoreclos ely atthe ser ver API, whose
foundation isthe http.Handler interface:
net/http
package http
type Handler interface {
ServeHTTP(w ResponseWriter, r *Request)
}
func ListenAndServe(address string, h Handler) error
The ListenAndServe func tionrequires a ser ver address, suchas "localhost:8000",and an
inst anceofthe Handler interface towhichall requests shouldbedispatch ed. Itrunsforever,
or unt i l theser ver fails (or fai lstostart)wit h an erro r,always non-ni l,whichitretur ns.
Im agine ane-commerce site wit h adat abas e mappingthe items for sale totheir prices in dol-
lars.The program below shows the simplestimaginableimp lementation.Itmodel s theinv en-
tory asamap typ e, database,towhichweve att ach edaServeHTTP method sothatitsat isfies
the http.Handler interface.The handler rangesoverthe map andprintsthe items.
gopl.io/ch7/http1
func main() {
db := database{"shoes": 50, "socks": 5}
log.Fatal(http.ListenAndServe("localhost:8000", db))
}
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
192 CHAPTER 7. INTERFACES
type dollars float32
func (d dollars) String() string { return fmt.Sprintf("$%.2f", d) }
type database map[string]dollars
func (db database) ServeHTTP(w http.ResponseWriter, req *http.Request) {
for item, price := range db {
fmt.Fprintf(w, "%s: %s\n", item, price)
}
}
If westart the ser ver,
$gobuild gopl.io/ch7/http1
$./http1 &
then connec t to itwit h the fetch prog ram from Sec tion 1.5 (oraweb brows er if you prefer),
we getthe fol low ing out put:
$gobuild gopl.io/ch1/fetch
$./fetch http://localhost:8000
shoes: $50.00
socks: $5.00
So far,the ser ver can onlylistits entire inv ent ory and will dothisfor every request, regardless
of URL. A more realist icser ver definesmultipledif ferentURLs, eachtrigger ingadif ferent
behavior.Letscal l theexist ing one /list andadd anot her one cal le d /price that rep ortsthe
pr ice ofasingleitem, specified asarequestparameter like /price?item=socks.
gopl.io/ch7/http2
func (db database) ServeHTTP(w http.ResponseWriter, req *http.Request) {
switch req.URL.Path {
case "/list":
for item, price := range db {
fmt.Fprintf(w, "%s: %s\n", item, price)
}
case "/price":
item := req.URL.Query().Get("item")
price, ok := db[item]
if !ok {
w.WriteHeader(http.StatusNotFound) // 404
fmt.Fprintf(w, "no such item: %q\n", item)
return
}
fmt.Fprintf(w, "%s\n", price)
default:
w.WriteHeader(http.StatusNotFound) // 404
fmt.Fprintf(w, "no such page: %s\n", req.URL)
}
}
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 7.7. THE HTTP.HANDLER INTERFACE 193
No w thehandler decides whatlog ic to exe cut e basedonthe pat h comp onent ofthe URL,
req.URL.Path.Ifthe handler doesntrecog nize thepat h, it rep ortsanHTTPerror tothe
clientbycal ling w.WriteHeader(http.StatusNotFound);thismustbedon e before writing
anytext to w.(In cidentally, http.ResponseWriter is another interface.Itaug ments
io.Writer with met hodsfor sendingHTTPrespons e he aders.) Equivalently, wecou lduse
the http.Error ut i lit y func tion:
msg := fmt.Sprintf("no such page: %s\n", req.URL)
http.Error(w, msg, http.StatusNotFound) // 404
Thecas e for /price callsthe URLs Query method toparse the HTTPrequestparametersasa
map, ormorepre cis ely,amultimapoftyp e url.Values (§6.2.1) fro m the net/url package. It
then nd s thefirs t item parameter andlooks upits price.Ifthe itemwasntfound,itrep orts
an erro r.
Heres an examplesessionwit h thenew server :
$gobuild gopl.io/ch7/http2
$gobuild gopl.io/ch1/fetch
$./http2 &
$./fetch http://localhost:8000/list
shoes: $50.00
socks: $5.00
$./fetch http://localhost:8000/price?item=socks
$5.00
$./fetch http://localhost:8000/price?item=shoes
$50.00
$./fetch http://localhost:8000/price?item=hat
no such item: "hat"
$./fetch http://localhost:8000/help
no such page: /help
Obviou sly wecou ldkeepaddingcas es to ServeHTTP,but inarealist icapp lic ation, itscon-
venienttodefine the log ic foreachcas e in a sep aratefunctionormet hod.Fur thermore,
re lated URLs may need simi lar logic; several image files may haveURLs ofthe for m
/images/*.png,for ins tance.For these reasons, net/http prov ides ServeMux,are quest
mu ltipl exer,tosimplif y theass oci ationbet weenURLs andhandlers. A ServeMux ag gregates a
collec tion of http.Handlersint o asingle http.Handler.Again, wesee thatdif ferenttyp es
satisf yingthe sameint erface are substitutabl e:the web ser ver can disp atch requests toany
http.Handler,regardlessofwhichcon crete typ e is behindit.
Fo r amorecomplex applic ation, several ServeMuxes may becomposedtohandlemore
intr icate dispatchingrequirements. Godoesnthaveacanonic al we b framewor k analogou s to
Ru bysRai lsorPyt honsDjango.Thisisnot tosay thatsuchframewor ksdontexist,but the
buildingblo cks in Gosstand ard librar y are flexibleenoug h that framewor ksare often
unnecessary.Fur thermore, alt hough framewor ksare convenientinthe early phas es of a
proj e ct, their addition alcomplexity canmakelon g er-term maintenanceharder.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
194 CHAPTER 7. INTERFACES
In the program below,wecre ate a ServeMux anduse ittoass oci atethe URLs wit h thecor-
resp ondinghandlersfor the /list and /price op erat ions,whichhavebeensplit int o separate
methods. Wethenuse the ServeMux as the main handler in the cal l to ListenAndServe.
gopl.io/ch7/http3
func main() {
db := database{"shoes": 50, "socks": 5}
mux := http.NewServeMux()
mux.Handle("/list", http.HandlerFunc(db.list))
mux.Handle("/price", http.HandlerFunc(db.price))
log.Fatal(http.ListenAndServe("localhost:8000", mux))
}
type database map[string]dollars
func (db database) list(w http.ResponseWriter, req *http.Request) {
for item, price := range db {
fmt.Fprintf(w, "%s: %s\n", item, price)
}
}
func (db database) price(w http.ResponseWriter, req *http.Request) {
item := req.URL.Query().Get("item")
price, ok := db[item]
if !ok {
w.WriteHeader(http.StatusNotFound) // 404
fmt.Fprintf(w, "no such item: %q\n", item)
return
}
fmt.Fprintf(w, "%s\n", price)
}
Letsfocus onthe two cal lsto mux.Handle that reg ister the handlers. Inthe rs t on e, db.list
is a met hod value (§6.4), thatis, a value oftyp e
func(w http.ResponseWriter, req *http.Request)
that, whencal le d,inv okesthe database.list method wit h thereceivervalue db.So db.list
is a functionthatimp lements handler-li kebeh avior,but since ithas nomet hods, itdoesntsat-
isfy the http.Handler interface andcantbepasseddirec tly to mux.Handle.
Theexpression http.HandlerFunc(db.list) is a conversion, not a functioncal l,since
http.HandlerFunc is a typ e.Ithas the fol low ing definition:
net/http
package http
type HandlerFunc func(w ResponseWriter, r *Request)
func (f HandlerFunc) ServeHTTP(w ResponseWriter, r *Request) {
f(w, r)
}
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 7.7. THE HTTP.HANDLER INTERFACE 195
HandlerFunc demon strates som e unusualfeaturesofGosint erface mechanism. Itisafunc-
tion typ e that has met hodsand sat isfies an interface, http.Handler.The beh avior of its
ServeHTTP method istocal l theunderly ing function. HandlerFunc is thu s an adapterthat
lets a functionvalue sat isf y an interface,where the functionand the int erfacessolemet hod
have the samesig nature. Ineffec t,thistrick lets a singletyp e such as database satisf y the
http.Handler interface several different ways: oncethrough its list method,oncethrough
its price method,and soon.
Becaus e register ingahandler thisway issocommon, ServeMux hasaconvenience met hod
called HandleFunc that does itfor us, sowecan simplif y thehandler reg ist rat ioncodetothis:
gopl.io/ch7/http3a
mux.HandleFunc("/list", db.list)
mux.HandleFunc("/price", db.price)
Itseasy tosee fro m thecodeabove how one wou ldcon str uct a program in whichthere are
twodif ferentweb ser vers, listeningondif ferentpor ts, definingdif ferentURLs, anddis-
patchingtodif ferenthandlers. Wewou ldjustcon str uct another ServeMux andmakeanother
call to ListenAndServe,perhaps conc urrently. But inmostprograms, one web ser ver is
plenty.Als o,itstypic al to define HTTPhandlersacrossmany files ofanapp lic ation, andit
wouldbeanuisance if the y al l hadtobeexplicitlyreg istered wit h theapp lic ations ServeMux
inst ance.
So,for convenience, net/http prov ides a global ServeMux inst ancecal le d DefaultServeMux
andpackage-le vel functions cal le d http.Handle and http.HandleFunc.Touse Default-
ServeMux as the ser versmain handler,weneedntpassitto ListenAndServe; nil wi l l do.
Theser versmain functioncan thenbesimplified to
gopl.io/ch7/http4
func main() {
db := database{"shoes": 50, "socks": 5}
http.HandleFunc("/list", db.list)
http.HandleFunc("/price", db.price)
log.Fatal(http.ListenAndServe("localhost:8000", nil))
}
Final ly, animp ortantreminder : as wemention edinSec tion 1.7, the web ser ver invo kes each
hand ler in a new goroutine,sohandlersmusttakepre caution s such as locki n g when accessing
var iables thatother goroutines, includingother requests tothe samehandler,may beaccess-
ing. Well tal k ab out con cur rency inthe next two chapt ers.
Exercis e 7.11: Addaddition alhandlerssothatclients can create, read, update, and delete
database ent ries. For example, a requestofthe for m /update?item=socks&price=6 wi l l
up datethe price ofaniteminthe inv ent ory and rep ort anerror ifthe itemdoesnot exist orif
theprice isinvalid.(Warning: thischange int roduces conc urrentvar iable updates.)
Exercis e 7.12: Change the handler for /list to print its out put asanHTMLtable,not text.
Yo u mayfind the html/template package(§4.6) useful.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
196 CHAPTER 7. INTERFACES
7.8. The error Int erface
Sincethe beg inningofthisbook, weve beenusingand cre ating values ofthe mysteriou s
predeclare d error type wit houtexplainingwhatitreallyis. Infac t, itsjustanint erface typ e
with a singlemet hod thatretur ns an error message:
type error interface {
Error() string
}
Thesimplestway tocre ate an error is bycal ling errors.New,whichretur nsanew error for
agiven erro r mess age . Theent ire errors packageisonlyfourlines long:
package errors
func New(text string) error { return &errorString{text} }
type errorString struct { text string }
func (e *errorString) Error() string { return e.text }
Theunderly ing typ e of errorString is a str uct,not a str ing , to pro tec t itsrepresent ation fro m
inadvertent(or premedit ate d)updates. And the reasonthatthe point ertyp e *errorString,
not errorString alone,sat isfies the error interface issothatevery cal l to New al locatesadis-
tinc t error inst ancethatisequ altonoother.Wewou ldnot wantadistinguishe d er ror such
as io.EOF to compare equ altoone thatmerelyhappene d to havethe samemessage .
fmt.Println(errors.New("EOF") == errors.New("EOF")) // "false"
Callsto errors.New arerel ative lyinf requentbecause theresaconvenientwrapperfunction,
fmt.Errorf,thatdoesstr ing for matting too.Weuseditseveral times in Chapt er5.
package fmt
import "errors"
func Errorf(format string, args ...interface{}) error {
return errors.New(Sprintf(format, args...))
}
Although *errorString maybethe simplesttyp e of error,itisfar fro m theonlyone.For
example, the syscall packageprovides Goslow-le vel systemcal l API. On many platfor ms, it
definesanumerictyp e Errno that sat isfies error,and onUnix platfor ms, Errnos Error
method doesalooku p in a table ofstr ings, as shown below :
package syscall
type Errno uintptr // operating system error code
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 7.9. EXAMPLE: EXPRESSION EVALUATOR197
var errors = [...]string{
1: "operation not permitted", // EPERM
2: "no such file or directory", // ENOENT
3: "no such process", // ESRCH
// ...
}
func (e Errno) Error() string {
if 0 <= int(e) && int(e) < len(errors) {
return errors[e]
}
return fmt.Sprintf("errno %d", e)
}
Thefol low ing statement cre atesanint erface value holdingthe Errno value 2,sig nifying the
POSIX ENOENT condit ion:
var err error = syscall.Errno(2)
fmt.Println(err.Error()) // "no such file or directory"
fmt.Println(err) // "no such file or directory"
Thevalue of err is shown graphic ally inFigure7.6.
Figure 7.6. An int erface value holdingasyscall.Errno integer.
Errno is anefficientrepresent ation of systemcal l er ror s draw n from a finite set, anditsat isfies
thestand ard error interface.Well see other typ es that sat isf y this int erface in Sec tion 7.11.
7.9. Example: Expression Evaluator
In thissec tion,well bui ld an evaluatorfor simplearithmeticexpressions.Well use anint er-
face, Expr,torepresent any expressioninthislangu age. For now,thisint erface needsno
methods, but well add som e later.
// An Expr is an arithmetic expression.
type Expr interface{}
Ourexpressionlangu agecon sists of floating-p ointlit erals; the binar y op erator s +, -, *,and /;
theunary operator s -x and +x;functioncal ls pow(x,y), sin(x),and sqrt(x);var iables such
as x and pi;and ofcours e parent hes es andstand ard operator precedence.All values areof
type float64.Hereare som e exampleexpressions:
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
198 CHAPTER 7. INTERFACES
sqrt(A / pi)
pow(x, 3) + pow(y, 3)
(F - 32) * 5 / 9
The ve con crete typ es belowrepresent par tic ularkinds ofexpression. A Var repres ents a ref-
erence toavar iable.(Well soonsee why itisexp orted.) A literal repres ents a floating-
pointcon stant.The unary and binary typesrepresent operator expressions wit h on e or two
op erands,whichcan beany kindof Expr.Acall repres ents a functioncal l; well restr ict its
fn eldto pow, sin,or sqrt.
gopl.io/ch7/eval
// A Var identifies a variable, e.g., x.
type Var string
// A literal is a numeric constant, e.g., 3.141.
type literal float64
// A unary represents a unary operator expression, e.g., -x.
type unary struct {
op rune // one of '+', '-'
xExpr
}
// A binary represents a binary operator expression, e.g., x+y.
type binary struct {
op rune // one of '+', '-', '*', '/'
x, y Expr
}
// A call represents a function call expression, e.g., sin(x).
type call struct {
fn string // one of "pow", "sin", "sqrt"
args []Expr
}
To evaluate anexpressioncontainingvar iables, well need an enviro nment that maps variable
namestovalues:
type Env map[Var]float64
Well als o ne e d each kindofexpressiontodefine an Eval method thatretur nsthe expressions
value in a given enviro nment.Since every expressionmustprovide thismet hod,weadd itto
the Expr interface.The packageexp ortsonlythe typ es Expr, Env,and Var;clients can use the
evaluatorwit houtaccesstothe other expressiontyp es.
type Expr interface {
// Eval returns the value of this Expr in the environment env.
Eval(env Env) float64
}
Thecon crete Eval methodsare shown below.The method for Var perfor msanenv iro nment
lo oku p,whichretur nszeroifthe var iable isnot define d,and the method for literal simply
returnsthe lit eral value.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 7.9. EXAMPLE: EXPRESSION EVALUATOR199
func (v Var) Eval(env Env) float64 {
return env[v]
}
func (l literal) Eval(_ Env) float64 {
return float64(l)
}
The Eval methodsfor unary and binary re cursive lyevaluate their operands,thenapp l y the
op erat ion op to them. Wedontcon sider divisions byzeroorinfinity tobeerror s,since the y
produce a result, albeit non-finite. Final ly, the method for call evaluatesthe arguments tothe
pow, sin,or sqrt func tion, thencal lsthe cor respondingfunction in the math package.
func (u unary) Eval(env Env) float64 {
switch u.op {
case '+':
return +u.x.Eval(env)
case '-':
return -u.x.Eval(env)
}
panic(fmt.Sprintf("unsupported unary operator: %q", u.op))
}
func (b binary) Eval(env Env) float64 {
switch b.op {
case '+':
return b.x.Eval(env) + b.y.Eval(env)
case '-':
return b.x.Eval(env) - b.y.Eval(env)
case '*':
return b.x.Eval(env) * b.y.Eval(env)
case '/':
return b.x.Eval(env) / b.y.Eval(env)
}
panic(fmt.Sprintf("unsupported binary operator: %q", b.op))
}
func (c call) Eval(env Env) float64 {
switch c.fn {
case "pow":
return math.Pow(c.args[0].Eval(env), c.args[1].Eval(env))
case "sin":
return math.Sin(c.args[0].Eval(env))
case "sqrt":
return math.Sqrt(c.args[0].Eval(env))
}
panic(fmt.Sprintf("unsupported function call: %s", c.fn))
}
Several ofthese met hodscan fail. For example, a call expressioncou ldhaveanunknown
func tionorthe wro ngnumberofarguments. Itsals o possible tocon str uct a unary or binary
expressionwit h an invalid operator suchas ! or < (although the Parse func tionmention ed
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
200 CHAPTER 7. INTERFACES
belowwill never do this). Thes e er ror s caus e Eval to panic. Other erro rs, like evaluating a
Var notpresent inthe env iro nment,merelycause Eval to retur n thewro ngresult. All ofthese
er ror s couldbedetec ted byins pec ting the Expr before evaluating it. That will bethe job ofthe
Check method,whichwewill showsoon, but rs t letstest Eval.
The TestEval func tionbelow isa testofthe evaluator. Itusesthe testing package, which
well explain in Chapt er11, but for now itsenoug h to knowthatcal ling t.Errorf reportsan
er ror.The functionloops overatable ofinp uts thatdefinesthree expressions and dif ferent
enviro nmentsfor eachone.The rs t expressioncomputesthe radius ofacirclegiven its area
A,the secon d comp utesthe sum ofthe cub esoftwo var iables x and y,and the thirdconvertsa
Fahren heittemperature F to Cel siu s.
func TestEval(t *testing.T) {
tests := []struct {
expr string
env Env
want string
}{
{"sqrt(A / pi)", Env{"A": 87616, "pi": math.Pi}, "167"},
{"pow(x, 3) + pow(y, 3)", Env{"x": 12, "y": 1}, "1729"},
{"pow(x, 3) + pow(y, 3)", Env{"x": 9, "y": 10}, "1729"},
{"5 / 9 * (F - 32)", Env{"F": -40}, "-40"},
{"5 / 9 * (F - 32)", Env{"F": 32}, "0"},
{"5 / 9 * (F - 32)", Env{"F": 212}, "100"},
}
var prevExpr string
for _, test := range tests {
// Print expr only when it changes.
if test.expr != prevExpr {
fmt.Printf("\n%s\n", test.expr)
prevExpr = test.expr
}
expr, err := Parse(test.expr)
if err != nil {
t.Error(err) // parse error
continue
}
got := fmt.Sprintf("%.6g", expr.Eval(test.env))
fmt.Printf("\t%v => %s\n", test.env, got)
if got != test.want {
t.Errorf("%s.Eval() in %s = %q, want %q\n",
test.expr, test.env, got, test.want)
}
}
}
Fo r each ent ryinthe table,the testparsesthe expression, evaluatesitinthe env iro nment,and
pr intsthe result. Wedonthavespace toshowthe Parse func tionhere, but youll nd itifyou
down loadthe packageusing go get.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 7.9. EXAMPLE: EXPRESSION EVALUATOR201
The go test command(§11.1) runsapackagestests:
$gotest -v gopl.io/ch7/eval
The -v flag lets ussee the print edout put of the test, whichisnor mal lysup pressedfor a suc-
cessf ultestlikethisone.Hereisthe out put of the tests fmt.Printf st atements:
sqrt(A / pi)
map[A:87616 pi:3.141592653589793] => 167
pow(x, 3) + pow(y, 3)
map[x:12 y:1] => 1729
map[x:9 y:10] => 1729
5/9*(F-32)
map[F:-40] => -40
map[F:32] => 0
map[F:212] => 100
Fo rtunate lythe inp uts sofar haveall beenwel l formed,but our luckisunlikelytolast. Evenin
interprete d languages, itiscommontoche ckthe syntaxfor stati c er ror s,thatis, mist akesthat
canbedetec ted wit houtrunningthe program. Bysep arat ingthe staticche cks fro m the
dy namic ones, wecan detec t er ror s so onerand per for m many che cks onlyonceins teadof
each time an expressionisevaluate d.
Letsadd anot her met hod tothe Expr interface.The Check method che cks for staticerror s in
an expressionsyntaxtre e.Well explain its vars parameter in a mom ent.
type Expr interface {
Eval(env Env) float64
// Check reports errors in this Expr and adds its Vars to the set.
Check(vars map[Var]bool) error
}
Thecon crete Check methodsare shown below.Evaluation of literal and Var cannot fai l,so
the Check methodsfor these typ es return nil.The methodsfor unary and binary rs t ch eck
that the operator isvalid,thenrec ursivelyche ckthe operands.Simi larly,the method for call
rs t ch ecksthatthe functionisknown and has the rig htnumberofarguments, thenrec ur-
sive lyche cks eachargument.
func (v Var) Check(vars map[Var]bool) error {
vars[v] = true
return nil
}
func (literal) Check(vars map[Var]bool) error {
return nil
}
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
202 CHAPTER 7. INTERFACES
func (u unary) Check(vars map[Var]bool) error {
if !strings.ContainsRune("+-", u.op) {
return fmt.Errorf("unexpected unary op %q", u.op)
}
return u.x.Check(vars)
}
func (b binary) Check(vars map[Var]bool) error {
if !strings.ContainsRune("+-*/", b.op) {
return fmt.Errorf("unexpected binary op %q", b.op)
}
if err := b.x.Check(vars); err != nil {
return err
}
return b.y.Check(vars)
}
func (c call) Check(vars map[Var]bool) error {
arity, ok := numParams[c.fn]
if !ok {
return fmt.Errorf("unknown function %q", c.fn)
}
if len(c.args) != arity {
return fmt.Errorf("call to %s has %d args, want %d",
c.fn, len(c.args), arity)
}
for _, arg := range c.args {
if err := arg.Check(vars); err != nil {
return err
}
}
return nil
}
var numParams = map[string]int{"pow": 2, "sin": 1, "sqrt": 1}
Weve liste d aselec tion of flawed inp uts andthe error s they elicit, in two gro ups. The Parse
func tion(notshown)rep ortssyntaxerror s andthe Check func tionrep ortssemanticerror s.
x%2 unexpected '%'
math.Pi unexpected '.'
!true unexpected '!'
"hello" unexpected '"'
log(10) unknown function "log"
sqrt(1, 2) call to sqrt has 2 args, want 1
Checksargument, a set of Vars, accumulates the set ofvar iable names found wit hin the
expression. Eachofthese var iables mustbepresent inthe env iro nment for evaluation tosuc-
ce e d.Thisset islog ical lythe re sult of the cal l to Check,but because the method isrec ursive, it
is moreconvenientfor Check to popu lateaset passedasaparameter.The clientmustprovide
an empt y set in the initial cal l.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 7.9. EXAMPLE: EXPRESSION EVALUATOR203
In Sec tion 3.2, weplott eda function f(x,y) that was xe d at comp ile time. Now thatwecan
pars e, ch eck,and evaluate expressions instr ings, wecan bui ld aweb app lic ationthatreceives
an expressionatrun timefro m theclientand plots the sur face of thatfunction. Wecan use
the vars settoche ckthatthe expressionisafunctionofonlytwo var iables, x and ythre e,
ac tually, since well provide r,the radius,asaconvenience.And well use the Check method to
rejec t ill-for med expressions beforeevaluation beg inssothatwedontrep eat those che cks dur-
ingthe 40,000 evaluation s (100&100 cells, eachwit h four corners) ofthe functionthatfol low.
The parseAndCheck func tioncom binesthese parsingand che cking steps:
gopl.io/ch7/surface
import "gopl.io/ch7/eval"
func parseAndCheck(s string) (eval.Expr, error) {
if s == "" {
return nil, fmt.Errorf("empty expression")
}
expr, err := eval.Parse(s)
if err != nil {
return nil, err
}
vars := make(map[eval.Var]bool)
if err := expr.Check(vars); err != nil {
return nil, err
}
for v := range vars {
if v != "x" && v != "y" && v != "r" {
return nil, fmt.Errorf("undefined variable: %s", v)
}
}
return expr, nil
}
To makethisaweb app lic ation, allweneed isthe plot func tionbelow,whichhas the fami liar
sig natureofan http.HandlerFunc:
func plot(w http.ResponseWriter, r *http.Request) {
r.ParseForm()
expr, err := parseAndCheck(r.Form.Get("expr"))
if err != nil {
http.Error(w, "bad expr: "+err.Error(), http.StatusBadRequest)
return
}
w.Header().Set("Content-Type", "image/svg+xml")
surface(w, func(x, y float64) float64 {
r:=math.Hypot(x, y) // distance from (0,0)
return expr.Eval(eval.Env{"x": x, "y": y, "r": r})
})
}
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
204 CHAPTER 7. INTERFACES
Figure 7.7. Thesur faces of three functions:(a) sin(-x)*pow(1.5,-r);
(b) pow(2,sin(y))*pow(2,sin(x))/12;(c) sin(x*y/10)/10.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 7.10. TYPE ASSERTIONS 205
The plot func tionparsesand che cks the expressionspecified inthe HTTPrequestand usesit
to cre ate ananony mou s func tionoftwo var iables. Theanony mou s func tionhas the samesig-
nature asthe xe d func tion f from the originalsur face-plott ing program, but it evaluatesthe
us er-supp lie d expression. Theenv iro nment defines x, y,and the radius r.Final ly, plot calls
surface,whichisjustthe main func tionfro m gopl.io/ch3/surface,modified totakethe
func tiontoplotand the out put io.Writer as parameters, insteadofusingthe xe d func tion f
and os.Stdout.Figure7.7 shows three sur faces produce d by the program.
Exercis e 7.13: AddaString method to Expr to prett y-print the syntaxtre e.Che ckthatthe
resu lts, whenparsedagain, yield an equivalenttre e.
Exercis e 7.14: Dene a newcon crete typ e that sat isfies the Expr interface andprovides a new
op erat ionsuchascomputing the minimum value ofits operands.Since the Parse func tion
do es notcre ate ins tances ofthisnew typ e,touse ityou will need tocon str uct a syntaxtre e
direc tly (or extendthe parser).
Exercis e 7.15: Wr ite a program thatreads a singleexpressionfro m thestand ard inp ut,
promptsthe usertoprovide values for any var iables, thenevaluatesthe expressioninthe
resu lting env iro nment.Handleall error s gracef ully.
Exercis e 7.16: Wr ite a web-b ased calc ulatorprogram.
7.10. Type Assertions
A ty peass ertion is anoperat ionapp lie d to anint erface value.Syntactic ally,itlooks like x.(T),
where x is anexpressionofanint erface typ e and T is a typ e,cal le d the ‘‘asserted’’ type.Atype
assertionche cks thatthe dynamic typ e of itsoperandmatch esthe ass erted typ e.
Thereare two possibi lit ies. First,ifthe ass erted typ e T is a con crete typ e,thenthe typ e asser-
tion che cks whether xsdynamic typ e is ident icalto T.Ifthische cksucce e ds, the resultofthe
type ass ertionis xsdynamic value,whose typ e is ofcours e T.Inother words,atyp e assertion
to a con crete typ e ext racts the con crete value fro m itsoperand. Ifthe che ckfai ls, thenthe
op erat ionpanics. For example:
var w io.Writer
w=os.Stdout
f:=w.(*os.File) // success: f == os.Stdout
c:=w.(*bytes.Buffer) // panic: interface holds *os.File, not *bytes.Buffer
Second,ifins teadthe ass erted typ e T is anint erface typ e,thenthe typ e assertionche cks
whet her xsdynamic typ e sati ses T.Ifthische cksucce e ds, the dynamic value isnot ext racte d;
theresultisstill anint erface value wit h thesametyp e andvalue comp onents, but the result
hasthe int erface typ e T.Inother words,atyp e assertiontoanint erface typ e ch angesthe typ e
of the expression, mak ingadif ferent(andusu allylarger) set ofmet hodsaccessible, but it
pres ervesthe dynamic typ e andvalue comp onentsinside the int erface value.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
206 CHAPTER 7. INTERFACES
Af ter the rs t type ass ertionbelow,bot h w and rw hold os.Stdout so eachhas a dynamic typ e
of *os.File,but w,an io.Writer,exp oses onlythe files Write method,where as rw exp oses
its Read method too.
var w io.Writer
w=os.Stdout
rw := w.(io.ReadWriter) // success: *os.File has both Read and Write
w=new(ByteCounter)
rw = w.(io.ReadWriter) // panic: *ByteCounter has no Read method
No matterwhattyp e was asserted,ifthe operandisanil int erface value,the typ e assertion
fai ls. A type ass ertiontoalessrestr ictiveint erface typ e (one wit h fe wer met hods) israrely
ne e ded, as it beh avesjustlike an assig nment, except in the nil cas e.
w=rw//io.ReadWriter is assignable to io.Writer
w=rw.(io.Writer) // fails only if rw == nil
Of ten were not sureofthe dynamic typ e of anint erface value,and wedliketotestwhether it
is som e partic ulartyp e.Ifthe typ e assertionapp earsinanassig nmentinwhichtwo results are
exp ected,suchasthe fol low ing declarat ions,the operat iondoesnot panic onfai lurebut
insteadretur ns an addition alsecon d resu lt, a boole an indic atingsuccess:
var w io.Writer = os.Stdout
f, ok := w.(*os.File) // success: ok, f == os.Stdout
b, ok := w.(*bytes.Buffer) // failure: !ok, b == nil
Thesecon d resu ltisconvent ion allyassig ned toavar iable named ok.Ifthe operat ionfai le d,
ok is false,and the rs t resu ltisequ altothe zerovalue ofthe ass erted typ e,whichinthis
exampleisanil *bytes.Buffer.
The ok resu ltisoften immediate lyusedtodecide whattodonext. Theextende d form ofthe
if st atement makes thisquite compact:
if f, ok := w.(*os.File); ok {
// ...use f...
}
Wh enthe operandofatyp e assertionisavar iable,rat her thaninv ent another namefor the
ne w lo cal variable, youll som etimessee the originalnamereused, shadowing the original, like
this:
if w, ok := w.(*os.File); ok {
// ...use w...
}
7.11. Discriminating Errorswith Type Assertions
Consider the set oferror s returned byfile operat ions inthe os package. I/O can failfor any
numb erofreasons,but three kinds offai lureoften mustbehandled dif ferently: file already
exists(forcre ate operat ions), file not found (forreadoperat ions), andper missiondenie d.The
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 7.11. DISCRIMINATING ERRORS WITH TYPE ASSERTIONS 207
os packageprovides these three helperfunctions toclassif y thefai lureindic ated byagiven
error value:
package os
func IsExist(err error) bool
func IsNotExist(err error) bool
func IsPermission(err error) bool
Anve imp lementation of oneofthese pre dic ates might che ckthatthe error message con-
tainsacer tain subst ring,
func IsNotExist(err error) bool {
// NOTE: not robust!
return strings.Contains(err.Error(), "file does not exist")
}
butbecause the log ic forhandlingI/O erro rscan varyfro m on e pl atfor m to another,this
approach isnot robustand the samefai luremay berep orted wit h avar ietyofdif ferenterror
mess ages. Che cking for subst rings oferror messagesmay beusefuldur ingtesting toens ure
that functions fai l in the exp ected manner,but itsinade quate for produc tioncode.
Amorereliableappro ach istorepresent str uctured error values usingadedic ated typ e.The
os packagedefinesatyp e called PathError to des crib e fai lures invo l vinganoperat ionona
file pat h, li ke Open or Delete,and a var iantcal le d LinkError to des crib e fai lures ofopera-
tion s invo l vingtwo file pat hs, like Symlink and Rename.Heres os.PathError:
package os
// PathError records an error and the operation and file path that caused it.
type PathError struct {
Op string
Path string
Err error
}
func (e *PathError) Error() string {
return e.Op + " " + e.Path + ": " + e.Err.Error()
}
Most clients areobliv iou s to PathError anddealwit h al l er ror s in a unifor m way bycal ling
their Error methods. Alt hough PathErrors Error method for msamessage bysimply con-
catenating the elds, PathErrorsstr ucturepreserves the underly ing componentsofthe error.
Clients thatneed todistinguish one kindoffai lurefro m anot her can use a typ e assertionto
detec t thespecifictyp e of the error ; thespecifictyp e prov ides moredet ailthanasimplestr ing .
_, err := os.Open("/no/such/file")
fmt.Println(err) // "open /no/such/file: No such file or directory"
fmt.Printf("%#v\n", err)
// Output:
// &os.PathError{Op:"open", Path:"/no/such/file", Err:0x2}
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
208 CHAPTER 7. INTERFACES
Thatshow the three helperfunctions work.For example, IsNotExist,shown below,rep orts
whet her an erro r is equ alto syscall.ENOENT (§7.8) ortothe distinguishe d er ror
os.ErrNotExist (s ee io.EOF in §5.4.2), orisa*PathError whos e underly ing error isone of
thos e two.
import (
"errors"
"syscall"
)
var ErrNotExist = errors.New("file does not exist")
// IsNotExist returns a boolean indicating whether the error is known to
// report that a file or directory does not exist. It is satisfied by
// ErrNotExist as well as some syscall errors.
func IsNotExist(err error) bool {
if pe, ok := err.(*PathError); ok {
err = pe.Err
}
return err == syscall.ENOENT || err == ErrNotExist
}
Andhere it is in action:
_, err := os.Open("/no/such/file")
fmt.Println(os.IsNotExist(err)) // "true"
Of course, PathErrorsstr uctureislostifthe error message iscom bine d into a largerstr ing ,
forins tance byacal l to fmt.Errorf.Error dis criminat ionmustusu allybedon e immediate ly
af ter the fai lingoperat ion, before an error ispro pagated tothe cal ler.
7.12. Querying Behaviorswith Int erfaceType Assertions
Thelog ic belowissimi lar tothe par t of the net/http we b server responsible for writing
HT TP he ader elds suchas "Content-type: text/html".The io.Writer w repres ents the
HT TP resp ons e;the bytes writt entoitare ult imate lysenttosom eon esweb brows er.
func writeHeader(w io.Writer, contentType string) error {
if _, err := w.Write([]byte("Content-Type: ")); err != nil {
return err
}
if _, err := w.Write([]byte(contentType)); err != nil {
return err
}
// ...
}
Becaus e the Write method requires a byteslice,and the value wewishtowrite isastr ing , a
[]byte(...) conv ersionisrequired.Thisconversionallocates memor y andmakes a copy,
butthe copyisthrow n away almostimmediate lyafter.Letspretend thatthisisacorepar t of
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 7.12. QUERYING BEHAVIORS WITH INTERFACE TYPE ASSERTIONS 209
theweb ser ver andthatour profilinghas reveale d that thismemor y al location isslowing it
down.Can weavoid allo cat ingmemor y here?
The io.Writer interface tel lsusonlyone fac t ab out the con crete typ e that w holds: thatbytes
maybewritt entoit. Ifwelookbehindthe cur tainsofthe net/http package, wesee thatthe
dy namic typ e that w holdsinthisprogram also has a WriteString method thatallowsstr ings
to beefficientlywritt entoit, avoidingthe need toallocateatemporar y copy.(Thismay seem
li keashotinthe dark, but a numberofimp ortanttyp es that sat isf y io.Writer also havea
WriteString method,including *bytes.Buffer, *os.File and *bufio.Writer.)
We c annot assume thatanarbit rar y io.Writer w also has the WriteString method.But we
candefine a newint erface thathas justthismet hod and use a typ e assertiontotestwhether
thedynamic typ e of w satisfies thisnew int erface.
// writeString writes s to w.
// If w has a WriteString method, it is invoked instead of w.Write.
func writeString(w io.Writer, s string) (n int, err error) {
type stringWriter interface {
WriteString(string) (n int, err error)
}
if sw, ok := w.(stringWriter); ok {
return sw.WriteString(s) // avoid a copy
}
return w.Write([]byte(s)) // allocate temporary copy
}
func writeHeader(w io.Writer, contentType string) error {
if _, err := writeString(w, "Content-Type: "); err != nil {
return err
}
if _, err := writeString(w, contentType); err != nil {
return err
}
// ...
}
To avoid rep eat ingourselves, weve mov edthe che ckint o theutilit y func tion writeString,
butitissousefulthatthe stand ard librar y prov ides itas io.WriteString.Itisthe recom-
mended way towrite a str ing toan io.Writer.
Wh atscur ious inthisexampleisthatthere isnostand ard int erface thatdefinesthe
WriteString method and specifies its required beh avior.Fur thermore, whether ornot a con-
cretetyp e satisfies the stringWriter interface isdeter mined onlybyits met hods, not byany
de clare d re lat ions hip bet weenitand the int erface typ e.Whatthismeans isthatthe technique
ab ove relies onthe assump tionthat if atyp e satisfies the int erface below, th en
WriteString(s) mu sthavethe sameeffec t as Write([]byte(s)).
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
210 CHAPTER 7. INTERFACES
interface {
io.Writer
WriteString(s string) (n int, err error)
}
Although io.WriteString do cuments its assump tion, few functions thatcal l it are likelyto
do cumentthatthe y to o make the sameassump tion. Definingamet hod ofapar tic ulartyp e is
taken as an imp licitass ent for a cer tain beh avioral cont rac t. Ne wcomers to Go, esp eci ally
thos e from a backg round instron gly typ edlangu ages, may nd thislackofexplicitint ent ion
unsett ling, but it israrelyaproblem in prac tice.Wit h theexception of the emp tyint erface
interface{},int erface typ es areseldomsat isfied byunintended coincidence.
The writeString func tionabove usesatyp e assertiontosee whether a value ofageneral
interface typ e also sat isfies a morespecificint erface typ e,and ifso, itusesthe beh avior s of the
sp ecificint erface.Thistechnique can beput togood use whether ornot the quer iedint erface
is stand ard like io.ReadWriter or user-define d li ke stringWriter.
Itsals o how fmt.Fprintf dist inguishesvalues thatsat isf y error or fmt.Stringer from all
ot her values. Wit hin fmt.Fprintf,there isa stepthatconvertsasingleoperandtoastr ing ,
somethinglikethis:
package fmt
func formatOneValue(x interface{}) string {
if err, ok := x.(error); ok {
return err.Error()
}
if str, ok := x.(Stringer); ok {
return str.String()
}
// ...all other types...
}
If x satisfies either ofthe two int erfaces, thatdeter mines the for matting ofthe value.Ifnot,the
defau ltcas e hand les allother typ es more orlessunifor mly usingreflec tion;well nd out how
in Chapt er12.
Again, thismakes the assump tionthatany typ e with a String method sat isfies the beh avioral
cont rac t of fmt.Stringer,whichistoretur n astr ing suitablefor printing .
7.13. Type Switches
Interfaces areusedintwo distinc t st yles. Inthe rs t st yle,exemp lified by io.Reader,
io.Writer, fmt.Stringer, sort.Interface, http.Handler,and error,anint erfacesmet h-
odsexpress the simi lar ities ofthe con crete typ es that sat isf y theint erface but hide the rep-
resent ation det ails and int rinsic operat ions ofthose con crete typ es. Theemp hasisisonthe
methods, not onthe con crete typ es.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 7.13. TYPE SWITCHES 211
Thesecon d st yle exploitsthe abi lit y of anint erface value toholdvalues ofavar ietyofcon crete
typesand con sidersthe int erface tobethe union of those typ es. Type ass ertions are usedto
discriminateamong these typ es dy namic ally and tre ateachcas e dif ferently. Inthissty le, the
emph asisisonthe con crete typ es that sat isf y theint erface,not onthe int erfacesmet hods(if
inde e d it has any), andthere isnohidingofinfor mat ion. Well des crib e interfaces usedthis
way as di scr imi nat eduni ons.
If youre fami liar wit h objec t-oriente d prog ramming, you may recog nize thes e twosty les as
subtypepolymor phism and ad hocpolymor phism,but you needntrememberthose ter ms. For
theremainder ofthischapt er, well present examples ofthe secon d st yle.
GosAPI for quer yinganSQL dat abas e,likethose ofother langu ages, lets uscle anly sep arate
thefixe d part ofaquer y from the var iable par ts. An exampleclientmig htlooklikethis:
import "database/sql"
func listTracks(db sql.DB, artist string, minYear, maxYear int) {
result, err := db.Exec(
"SELECT * FROM tracks WHERE artist = ? AND ? <= year AND year <= ?",
artist, minYear, maxYear)
// ...
}
The Exec method replaces each '?' in the quer y st ringwit h an SQLlit eral denot ing the cor-
resp ondingargumentvalue,whichmay beaboole an, anumber, a str ing , or nil.Con str uct-
ingquer ies this way helps avoidSQL inj e ction att acks, in whichanadversary takes cont rol of
thequer y by exploit ing imp rop erquotation of inp utdat a. Wi thin Exec,wemig ht nd a func-
tion likethe one below,whichconvertseachargumentvalue toits literal SQLnot ation.
func sqlQuote(x interface{}) string {
if x == nil {
return "NULL"
}else if _, ok := x.(int); ok {
return fmt.Sprintf("%d", x)
}else if _, ok := x.(uint); ok {
return fmt.Sprintf("%d", x)
}else if b, ok := x.(bool); ok {
if b {
return "TRUE"
}
return "FALSE"
}else if s, ok := x.(string); ok {
return sqlQuoteString(s) // (not shown)
}else {
panic(fmt.Sprintf("unexpected type %T: %v", x, x))
}
}
A switch st atement simplifies an if-else ch ain thatper for msaser ies of value equ ality tests.
An analogou s ty peswitch st atement simplifies an if-else ch ain oftyp e assertions.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
212 CHAPTER 7. INTERFACES
In its simplestfor m,atyp e sw itch looks like anordinar y sw itch statement inwhichthe oper-
andis x.(type)thatslit eral lythe key word typeandeachcas e hasone ormoretyp es. A
type switch enables a multi-way branch bas edonthe int erface valuesdynamic typ e.The nil
case match esif x==nil,and the default case match esifnoother cas e do es. Atyp e sw itch
for sqlQuote wouldhavethese cas es:
switch x.(type) {
case nil: // ...
case int, uint: // ...
case bool: // ...
case string: // ...
default: // ...
}
As wit h an ordinar y sw itch statement (§1.8), cas es arecon sidered inorder and, whenamatch
is found,the cas esbodyisexe cut ed. Cas e order becom essig nificant whenone ormorecas e
typesare int erfaces, since thenthere isapossibi lit y of two cas es matching. The position of the
default case rel ative tothe othersisimmater ial.No fallthrough is allow ed.
No tice thatinthe originalfunction, the log ic forthe bool and string casesneedsaccessto
thevalue extrac ted bythe typ e assertion. Since thisistypic al,the typ e sw itch statement has an
extende d form that binds the ext racte d value toanew var iable wit hin eachcas e:
switch x := x.(type) { /* ... */ }
Here weve cal le d thenew var iables x to o;aswit h type ass ertions,reuse ofvar iable names is
common.Likeaswitch st atement,atyp e sw itch imp licitlycre atesalexic al block, sothe dec-
larat ionofthe newvar iable cal le d x do es notconflic t with a var iable x in an out erblo ck. Each
case also imp licitlycre atesasep aratelexic al block.
Re writ ing sqlQuote to use the extende d form oftyp e sw itch makes itsig nificant lycle arer :
func sqlQuote(x interface{}) string {
switch x := x.(type) {
case nil:
return "NULL"
case int, uint:
return fmt.Sprintf("%d", x) // x has type interface{} here.
case bool:
if x {
return "TRUE"
}
return "FALSE"
case string:
return sqlQuoteString(x) // (not shown)
default:
panic(fmt.Sprintf("unexpected type %T: %v", x, x))
}
}
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 7.14. EXAMPLE: TOKEN-BASED XML DECODING 213
In thisversion, wit hin the blo ckofeachsingle-t ypecas e,the var iable x hasthe sametyp e as
thecas e.For ins tance, x hastyp e bool within the bool case and string within the string
case.Inall other cas es, x hasthe (interface) typ e of the switch op erand, whichis inter-
face{} in thisexample. Whenthe sameactionisrequired for multiplecas es, li ke int and
uint,the typ e sw itch makes iteasy tocom bine them.
Although sqlQuote accepts an argumentofany typ e,the functionrunstocomplet iononlyif
theargumentstyp e match esone ofthe cas es in the typ e sw itch;other wis e it panics wit h an
‘‘unexp ected typ e’’ mess age . Although the typ e of x is interface{},wecon sider ita
di scr imi nat eduni on of int, uint, bool, string,and nil.
7.14. Example: To ken-Based XML Decoding
Section4.5 showe d howtodecodeJSONdocuments into Godat a st ruc tures wit h the Marshal
and Unmarshal func tions fro m the encoding/json package. The encoding/xml package
prov ides a similarAPI.Thisappro ach isconvenientwhenwewanttocon str uct a represen-
tation of the documenttre e,but thatsunnecessary for manyprograms. The encoding/xml
packageals o prov ides a lower-le vel tok en-bas ed APIfor decodingXML. Inthe token-b ased
st yle,the parsercon sumes the inp utand pro duces a streamoftokens, primarily offour
kind sStartElement, EndElement, CharData,and Commenteach beingacon crete typ e in
the encoding/xml package. Eachcal l to (*xml.Decoder).Token returnsatoken.
Therelevantpar ts of the API are shown here:
encoding/xml
package xml
type Name struct {
Local string // e.g., "Title" or "id"
}
type Attr struct { // e.g., name="value"
Name Name
Value string
}
// A Token includes StartElement, EndElement, CharData,
// and Comment, plus a few esoteric types (not shown).
type Token interface{}
type StartElement struct { // e.g., <name>
Name Name
Attr []Attr
}
type EndElement struct { Name Name } // e.g., </name>
type CharData []byte // e.g., <p>CharData</p>
type Comment []byte // e.g., <!-- Comment -->
type Decoder struct{ /* ... */ }
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
214 CHAPTER 7. INTERFACES
func NewDecoder(io.Reader) *Decoder
func (*Decoder) Token() (Token, error) // returns next Token in sequence
The Token interface,whichhas nomet hods, isals o an exampleofadis criminated union. The
purpos e of a tradition alint erface like io.Reader is tohide detai lsofthe con crete typ es that
satisf y it sothatnew imp lementation s canbecre ate d;eachcon crete typ e is tre ate d unifor mly.
By contrast, the set ofcon crete typ es that sat isf y adis criminated unionisfixe d by the desig n
andexp osed,not hidden. Discriminated uniontyp es have few met hods; functions thatoper-
ateonthemare express edasa set ofcas es usingatyp e sw itch,wit h dif ferentlog ic in eachcas e.
The xmlselect prog ram belowext racts andprintsthe text found beneath cer tain elements in
an XML document tre e.Usingthe API above , it can do its job inasinglepassoverthe inp ut
withoutevermater ializingthe tre e.
gopl.io/ch7/xmlselect
// Xmlselect prints the text of selected elements of an XML document.
package main
import (
"encoding/xml"
"fmt"
"io"
"os"
"strings"
)
func main() {
dec := xml.NewDecoder(os.Stdin)
var stack []string // stack of element names
for {
tok, err := dec.Token()
if err == io.EOF {
break
}else if err != nil {
fmt.Fprintf(os.Stderr, "xmlselect: %v\n", err)
os.Exit(1)
}
switch tok := tok.(type) {
case xml.StartElement:
stack = append(stack, tok.Name.Local) // push
case xml.EndElement:
stack = stack[:len(stack)-1] // pop
case xml.CharData:
if containsAll(stack, os.Args[1:]) {
fmt.Printf("%s: %s\n", strings.Join(stack, " "), tok)
}
}
}
}
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 7.14. EXAMPLE: TOKEN-BASED XML DECODING 215
// containsAll reports whether x contains the elements of y, in order.
func containsAll(x, y []string) bool {
for len(y) <= len(x) {
if len(y) == 0 {
return true
}
if x[0] == y[0] {
y=y[1:]
}
x=x[1:]
}
return false
}
Each timethe loopin main encounters a StartElement,itpushesthe elementsnameont o a
st ack,and for each EndElement it popsthe namefro m thestack.The API guarante esthatthe
sequence of StartElement and EndElement tokens will bepro perly match ed, eveninill-
formed documents. Commentsare ignored.When xmlselect encounters a CharData,it
pr intsthe text onlyifthe stack containsall the elements named bythe command-lineargu-
ments, in order.
Thecommand below printsthe text ofany h2 elements appear ingbeneath two level s of div
elements. Its inputisthe XML specification,its elf an XML document.
$gobuild gopl.io/ch1/fetch
$./fetch http://www.w3.org/TR/2006/REC-xml11-20060816 |
./xmlselect div div h2
html body div div h2: 1 Introduction
html body div div h2: 2 Documents
html body div div h2: 3 Logical Structures
html body div div h2: 4 Physical Structures
html body div div h2: 5 Conformance
html body div div h2: 6 Notation
html body div div h2: A References
html body div div h2: B Definitions for Character Normalization
...
Exercis e 7.17: Extend xmlselect so thatelements may beselec ted not justbyname, but by
their attr ibutestoo,inthe manner ofCSS, sothat, for ins tance,anelementlike
<div id="page" class="wide"> couldbeselec ted byamatching id or class as wel l as its
name.
Exercis e 7.18: Usingthe token-b ased decoder API, write a program thatwill readanarbit rar y
XML document and con str uct a tre e of gener ic no des thatrepresentsit. Nodes areoftwo
kind s: CharData no des represent text str ings, and Element no des represent named elements
andtheir attr ibutes. Eachelementnodehas a slice ofchi ld no des.
Yo u mayfind the fol low ing declarat ions helpf ul.
import "encoding/xml"
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
216 CHAPTER 7. INTERFACES
type Node interface{} // CharData or *Element
type CharData string
type Element struct {
Type xml.Name
Attr []xml.Attr
Children []Node
}
7.15. A Fe w WordsofAdvice
Wh endesig ninganew package, nov ice Goprogrammersoften start bycre ating a set ofint er-
faces andonlylater define the con crete typ es that sat isf y them. Thisappro ach results in many
interfaces, eachofwhichhas onlyasingleimp lementation.Dontdothat. Suchint erfaces are
unnecessary abstrac tions;the y also havearun-t imecost. You can restr ict whichmet hodsofa
type orfields ofastr uct are visible outside a packageusingthe exp ort mechanism (§6.6).
Interfaces areonlyneeded whenthere are two ormorecon crete typ es that mustbedealt wit h
in a unifor m way.
We makeanexception tothisrulewhenanint erface issat isfied byasinglecon crete typ e but
that typ e cannot liveinthe samepackageasthe int erface because ofits dependencies. Inthat
case, an int erface isagood way todecoupletwo packages.
Becaus e interfaces areused in Go onlywhenthe y aresat isfied bytwo ormoretyp es, they
ne cessarily abstrac t away fro m thedet ails ofany par tic ularimp lementation.The resultis
smal ler interfaces wit h fe wer,simpler met hods, often justone aswit h io.Writer or
fmt.Stringer.Small int erfaces areeasier tosat isf y when new typ es come along . Agood rule
of thumb for int erface designis askonlyfor whatyou need.
Thiscon cludes our tourofmet hodsand int erfaces. Gohas gre atsup por t forthe obj e ct-
or iente d st yle ofprogramming, but thisdoesnot meanyou need touse itexc lusively. Not
ever ythingneed beanobj e ct; stand alone functions havetheir place,asdounenc apsulated
data typ es. Observethattoget her,the examples in the rs t ve chapt ers ofthisbookcal l no
more thantwo dozen met hods, like input.Scan,asopp osedtoordinar y func tioncal lslike
fmt.Printf.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
8
Goroutines and Channels
Conc urrentprogramming, the expressionofaprogram as a comp osition of several
autonomous activ ities, has never beenmoreimp ortantthanitistoday.Web ser vershandle
re quests for thous and s of clients at once. Tablet andphone appsrender animat ions inthe user
interface whi le simultane ously per for mingcomputation and networ k re quests in the back-
ground.Eventradition albatch pro blemsre adsom e data,compute, write som e outputus e
conc urrency tohide the latency ofI/O operat ions and toexploit a moder n comp utersmany
processors,whichevery yeargrow in numberbut not inspeed.
Go enables two sty les ofcon cur rentprogramming. Thischapt erpresentsgoroutinesand
ch annel s,whichsup por t commu nicating sequent ial pro cesses or CSP,amodel ofcon cur rency
in whichvalues arepassedbet weenindep endentactiv ities (goroutines) but var iables arefor
themostpar t confine d to a singleactiv ity.Chapt er9 cov ers som e aspectsofthe more tradi-
tion almodel of sh are d memory multithre ading,whichwill befami liar if youve usedthreads in
ot her mainst reamlangu ages. Chapt er9 als o points out som e importanthazards and pit fal lsof
conc urrentprogrammingthatwewontdelve int o in thischapt er.
Ev enthoug h Gossup por t forcon cur rency isone ofits gre atstrengt hs, reasoningabout con-
currentprogramsisinherentlyharder thanabout sequential ones, andint uit ion s acquired
from sequential programmingmay at times lead usast ray.Ifthisisyourfirs t encounterwit h
conc urrency,werecommend spendingalit tle extra timethin kingabout the examples in these
twochapt ers.
8.1. Goroutines
In Go, eachcon cur rentlyexe cut ing activ ity iscal le d a goro utine.Con sider a program thathas
twofunctions,one thatdoessom e comp utation and one thatwritessom e output,and assume
that neither functioncal lsthe other.Asequential program may cal l on e func tionand then
217
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
218 CHAPTER 8. GOROUTINES AND CHANNELS
call the other,but inaconcur rent prog ram with two ormoregoroutines, cal lsto both func tions
canbeactive at the sametime. Well see suchaprogram in a mom ent.
If you haveusedoperat ingsystemthreads orthreads inother langu ages, thenyou can assume
fornow thatagoroutine issimi lar toathread, and youll beabletowrite cor rec t prog rams.
Thedif ferences bet weenthreads and goroutinesare ess ent ial lyquant itative , notqualitative ,
andwill bedes crib ed in Sec tion 9.8.
Wh enaprogram starts, its onlygoroutine isthe one thatcal lsthe main func tion, sowecal l it
the main goroutine.New goroutinesare cre ate d by the go st atement.Syntactic ally,ago st ate-
ment isanordinar y func tionormet hod cal l prefixe d by the key word go.Ago st atement
caus esthe functiontobecal le d in a newly cre ate d goro utine.The go st atement its elf com-
pletes immediate ly:
f() // call f(); wait for it to return
go f() // create a new goroutine that calls f(); don'twait
In the examplebelow,the main goroutine computesthe 45t h Fibonacci number. Since ituses
theter r ibly inefficientrec ursivealgor it hm, itrunsfor anappre ciabletime, dur ingwhichwed
li ketoprovide the userwit h avisualindic ationthatthe program isstill running, bydispl aying
an animated textual ‘‘spinner.’’
gopl.io/ch8/spinner
func main() {
go spinner(100 * time.Millisecond)
const n = 45
fibN := fib(n) // slow
fmt.Printf("\rFibonacci(%d) = %d\n", n, fibN)
}
func spinner(delay time.Duration) {
for {
for _, r := range `-\|/` {
fmt.Printf("\r%c", r)
time.Sleep(delay)
}
}
}
func fib(x int) int {
if x < 2 {
return x
}
return fib(x-1) + fib(x-2)
}
Af ter several secon dsofanimat ion, the fib(45) call retur nsand the main func tionprintsits
resu lt:
Fibonacci(45) = 1134903170
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 8.2. EXAMPLE: CONCURRENT CLOCK SERVER 219
The main func tionthenretur ns. Whenthishappens,all goroutinesare abr uptlyter minated
andthe program exits. Other thanbyretur ningfro m main or exiting the program, there isno
prog rammat ic way for onegoroutine tostopanother,but aswewill see later,there are ways to
communic atewit h agoroutine torequestthatitstopits elf.
No tice how the program isexpress edasthe composition of two aut onom ous activ ities, spin-
ningand Fib onacci comp utation.Eachiswritt enasasep aratefunctionbut bot h make
prog resscon cur rently.
8.2. Example: ConcurrentClock Server
Networ kingisanatural domain in whichtouse con cur rency since ser verstypic ally handle
many conne ction s from their clients at once, eachclientbeingess ent ial lyindep endentofthe
ot hers. Inthissec tion,well int roduce the net package, whichprovides the componentsfor
buildingnet wor ked clientand ser ver programsthatcommunic ateoverTCP,UDP,orUnix
domain sockets. The net/http packageweve beenusingsince Chapt er1 isbui ltontop of
func tions fro m the net package.
Ourfirs t exampleisasequential clo ckser ver thatwritesthe cur renttimetothe clientonceper
second:
gopl.io/ch8/clock1
// Clock1 is a TCP server that periodically writes the time.
package main
import (
"io"
"log"
"net"
"time"
)
func main() {
listener, err := net.Listen("tcp", "localhost:8000")
if err != nil {
log.Fatal(err)
}
for {
conn, err := listener.Accept()
if err != nil {
log.Print(err) // e.g., connection aborted
continue
}
handleConn(conn) // handle one connection at a time
}
}
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
220 CHAPTER 8. GOROUTINES AND CHANNELS
func handleConn(c net.Conn) {
defer c.Close()
for {
_, err := io.WriteString(c, time.Now().Format("15:04:05\n"))
if err != nil {
return // e.g., client disconnected
}
time.Sleep(1 * time.Second)
}
}
The Listen func tioncre atesanet.Listener,anobj e ctthatlistens for incomingconne ction s
on a net wor k port,inthiscas e TCPpor t localhost:8000.The listeners Accept method
blocks until anincomingconne ction requestismade,thenretur nsanet.Conn objec t rep-
resent ing the conne ction.
The handleConn func tionhandles one completeclientconne ction.Inaloop, itwritesthe cur-
rent time, time.Now(),tothe client. Since net.Conn satisfies the io.Writer interface,wecan
wr ite direc tly toit. Theloopend s when the write fai ls, mostlikelybecause the clienthas dis-
connec ted,atwhichpoint handleConn clos es itsside ofthe conne ction usingadefer red cal l to
Close andgoesbacktowaiting for another connec tion request.
The time.Time.Format method provides a way tofor mat dateand timeinfor mat ionby
example. Its argumentisatempl ate indic atinghow tofor mat a reference time, specifically
Mon Jan 2 03:04:05PM 2006 UTC-0700.The reference timehas eight components(dayofthe
we ek, month,day ofthe mont h,and soon). Any col lec tion of themcan appear in the Format
st ringinany order andinanumberoffor mats; the selec ted componentsofthe dateand time
wi l l be displ aye d in the selec ted for mats. Hereweare justusingthe hour,minut e , andsecon d
of the time. The time packagedefinestempl atesfor manystand ard timefor mats, suchas
time.RFC1123.The samemechanism isusedinreverse whenparsingatimeusing
time.Parse.
To conne cttothe ser ver,well need a clientprogram suchas nc (‘‘netc at’’), a stand ard utilit y
prog ram formanipu lat ingnet wor k connec tion s:
$gobuild gopl.io/ch8/clock1
$./clock1 &
$nclocalhost 8000
13:58:54
13:58:55
13:58:56
13:58:57
^C
Theclientdispl ays the timesentbythe ser ver eachsecon d until weint err upt the clientwit h
Cont rol-C, whichonUnix systems isech oed as ^C by the shell. If nc or netcat is not ins tal le d
on yoursystem, you can use telnet or thissimpleGoversionof netcat that uses net.Dial
to conne cttoaTCP ser ver :
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 8.2. EXAMPLE: CONCURRENT CLOCK SERVER 221
gopl.io/ch8/netcat1
// Netcat1 is a read-only TCP client.
package main
import (
"io"
"log"
"net"
"os"
)
func main() {
conn, err := net.Dial("tcp", "localhost:8000")
if err != nil {
log.Fatal(err)
}
defer conn.Close()
mustCopy(os.Stdout, conn)
}
func mustCopy(dst io.Writer, src io.Reader) {
if _, err := io.Copy(dst, src); err != nil {
log.Fatal(err)
}
}
Thisprogram reads dat a from the conne ction and writesittothe stand ard out put unt i l an
end-of-file condit ion or anerror occ urs. The mustCopy func tionisautilit y us edinseveral
examples in thissec tion.Letsrun two clients at the sametimeondif ferentter minals, one
shown tothe lef t andone tothe rig ht:
$gobuild gopl.io/ch8/netcat1
$./netcat1
13:58:54 $./netcat1
13:58:55
13:58:56
^C
13:58:57
13:58:58
13:58:59
^C
$killall clock1
The killall commandisaUnix utilit y that killsall pro cesseswit h thegiven name.
Thesecon d clientmustwaitunt i l thefirs t clientis finishe d becaus e theser ver is sequ ent ial;it
de alswit h on lyone clientatatime. Justone small change isneeded tomakethe ser ver con-
current: addingthe go ke yword tothe cal l to handleConn caus eseachcal l to run in its own
goro utine.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
222 CHAPTER 8. GOROUTINES AND CHANNELS
gopl.io/ch8/clock2
for {
conn, err := listener.Accept()
if err != nil {
log.Print(err) // e.g., connection aborted
continue
}
go handleConn(conn) // handle connections concurrently
}
No w,multipleclients can receive the time at once:
$gobuild gopl.io/ch8/clock2
$./clock2 &
$gobuild gopl.io/ch8/netcat1
$./netcat1
14:02:54 $./netcat1
14:02:55 14:02:55
14:02:56 14:02:56
14:02:57 ^C
14:02:58
14:02:59 $./netcat1
14:03:00 14:03:00
14:03:01 14:03:01
^C 14:03:02
^C
$killall clock2
Exercis e 8.1: Mo dif y clock2 to acceptapor t numb er, and write a program, clockwall,that
ac ts as a clientofseveral clo ckser versatonce, readingthe times fro m each one and displ aying
theresults in a table,akintothe wal l of clo cks seeninsom e bu sinessoffices. Ifyou have
accesstogeographic ally distr ibute d comp uters,run inst ances remot ely ; ot her wis e runlocal
inst ances ondif ferentpor tswit h faketimezon es.
$TZ=US/Eastern ./clock2 -port 8010 &
$TZ=Asia/Tokyo ./clock2 -port 8020 &
$TZ=Europe/London ./clock2 -port 8030 &
$clockwall NewYork=localhost:8010 London=localhost:8020 Tokyo=localhost:8030
Exercis e 8.2: Implementacon cur rentFileTransfer Pro tocol (FTP) ser ver.The ser ver should
interpretcommand s from eachclientsuchas cd to change direc tor y, ls to listadirec tor y, get
to sendthe contentsofafile,and close to clos e theconne ction.You can use the stand ard ftp
command as the client, orwrite yourown.
8.3. Example: ConcurrentEcho Server
Theclo ckser ver usedone goroutine per connec tion.Inthissec tion,well bui ld an ech o server
that usesmultiplegoroutinesper connec tion.Mostech o serversmerelywrite whate ver the y
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 8.3. EXAMPLE: CONCURRENT ECHO SERVER 223
re ad, whichcan bedon e with this trivialversionof handleConn:
func handleConn(c net.Conn) {
io.Copy(c, c) // NOTE: ignoring errors
c.Close()
}
Amoreint erest ing ech o server might simulatethe reverb erations ofarealech o,wit h the
resp ons e loud at rs t ("HELLO!"), thenmoderate("Hello!")after a delay, thenquiet
("hello!")beforefadingtonot hing, as in thisversionof handleConn:
gopl.io/ch8/reverb1
func echo(c net.Conn, shout string, delay time.Duration) {
fmt.Fprintln(c, "\t", strings.ToUpper(shout))
time.Sleep(delay)
fmt.Fprintln(c, "\t", shout)
time.Sleep(delay)
fmt.Fprintln(c, "\t", strings.ToLower(shout))
}
func handleConn(c net.Conn) {
input := bufio.NewScanner(c)
for input.Scan() {
echo(c, input.Text(), 1*time.Second)
}
// NOTE: ignoring potential errors from input.Err()
c.Close()
}
Well need toupg rade ourclientprogram sothatitsends ter minal inputtothe ser ver whi le
also copying the ser ver respons e to the out put,whichpresentsanother opp ortunity touse
conc urrency :
gopl.io/ch8/netcat2
func main() {
conn, err := net.Dial("tcp", "localhost:8000")
if err != nil {
log.Fatal(err)
}
defer conn.Close()
go mustCopy(os.Stdout, conn)
mustCopy(conn, os.Stdin)
}
Whilethe main goroutine reads the stand ard inp utand sends ittothe ser ver,asecon d
goro utine reads and printsthe ser versrespons e.Whenthe main goroutine encount ers the
endofthe inp ut, for example, after the usertyp es Cont rol-D (^D)atthe ter minal (orthe
equivalentControl-Z onMicros oft Windows), the program stops, evenifthe other goroutine
st i l l haswor k to do. (Well see how tomakethe program waitfor bot h sides to finish once
weve int roduce d ch annel s in Sec tion 8.4.1.)
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
224 CHAPTER 8. GOROUTINES AND CHANNELS
In the sessionbelow,the clientsinp utislef t-aligne d andthe ser versrespons esare indente d.
Theclientshouts at the ech o server three times:
$gobuild gopl.io/ch8/reverb1
$./reverb1 &
$gobuild gopl.io/ch8/netcat2
$./netcat2
Hello?
HELLO?
Hello?
hello?
Is there anybody there?
IS THERE ANYBODY THERE?
Yooo-hooo!
Is there anybody there?
is there anybody there?
YOOO-HOOO!
Yooo-hooo!
yooo-hooo!
^D
$killall reverb1
No tice thatthe thirdshout fro m theclientisnot dealt wit h until the secon d shout has petered
out, whichisnot ver y re alist ic. A re alech o wouldcon sistofthe comp ositi on of the three inde-
pendentshouts. Tosimulateit, well need moregoroutines. Again, allweneed todoisadd
the go ke yword , this timetothe cal l to echo:
gopl.io/ch8/reverb2
func handleConn(c net.Conn) {
input := bufio.NewScanner(c)
for input.Scan() {
go echo(c, input.Text(), 1*time.Second)
}
// NOTE: ignoring potential errors from input.Err()
c.Close()
}
Thearguments tothe functionstarted by go areevaluate d when the go st atement its elf isexe-
cute d;thu s input.Text() is evaluate d in the main goroutine.
No w theech oes arecon cur rentand overl ap in time:
$gobuild gopl.io/ch8/reverb2
$./reverb2 &
$./netcat2
Is there anybody there?
IS THERE ANYBODY THERE?
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 8.4. CHANNELS 225
Yooo-hooo!
Is there anybody there?
YOOO-HOOO!
is there anybody there?
Yooo-hooo!
yooo-hooo!
^D
$killall reverb2
Al l that was required tomakethe ser ver use con cur rency,not justtohandleconne ction s from
mu ltipleclients but evenwit hin a singleconne ction,was the ins ertionoftwo go ke yword s.
Ho wever in addingthese key words,wehad tocon sider caref ullythatitissafetocal l methods
of net.Conn conc urrently, whichisnot truefor mosttyp es. Well dis cussthe cruci al concept
of concur rency saf ety in the next chapt er.
8.4. Channels
If goroutinesare the activ ities ofacon cur rentGoprogram, ch ann els arethe conne ction s
betweenthem. A ch annel isacommunic ationmechanism thatlets one goroutine sendvalues
to another goroutine.Eachchannel isacon duit for values ofapar tic ulartyp e,cal le d the
ch annels el ement type.The typ e of a channel whose elements havetyp e int is writt en
chan int.
To cre ate a channel,weuse the bui lt-in make func tion:
ch := make(chan int) // ch has type 'chan int'
As wit h maps, a channel isareference to the dat a st ruc turecre ate d by make.Whenwecopya
ch annel orpassone asanargumenttoafunction, weare copying a reference,socal ler and
callee refer tothe samedat a st ruc ture. Aswit h ot her reference typ es, thezerovalue ofachan-
nel is nil.
Tw o ch annel s of the sametyp e maybecompare d using ==.The comparisonistrueifbot h are
references tothe samechannel dat a st ruc ture. A ch annel may also becompare d to nil.
Achannel has two princip aloperat ions, send and re ceive,col lec tive lyknown as
commu nicati ons.Asend statement transmits a value fro m on e goro utine,through the chan-
nel,toanother goroutine exe cut ing a cor respondingreceive expression. Bot h op erat ions are
wr itt enusingthe <- op erator.Inasendstatement,the <- separates the channel and value op-
erands.Inareceive expression, <- precedes the channel operand. A re ceive expressionwhose
resu ltisnot usedisavalid statement.
ch <- x // a send statement
x=<-ch // a receive expression in an assignment statement
<-ch // areceive statement; result is discarded
Channel s supp ort a thirdoperat ion, cl ose,whichsets a flag indic atingthatnomorevalues will
ever besentonthischannel;subsequentatt emp tstosendwill panic. Receive operat ions ona
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
226 CHAPTER 8. GOROUTINES AND CHANNELS
clos edchannel yield the values thathavebeensentunt i l no more values arelef t; anyreceive
op erat ions there after comp leteimmediate lyand yield the zerovalue ofthe channelselement
type.
To clos e achannel,wecal l thebui lt-in close func tion:
close(ch)
Achannel cre ate d with a simplecal l to make is cal le d an unbuffered ch annel,but make accepts
an opt ion alsecon d argument, an integercal le d thechannels capacity.Ifthe cap acity isnon-
zero, make createsabuffered ch annel.
ch = make(chan int) // unbuffered channel
ch = make(chan int, 0) // unbuffered channel
ch = make(chan int, 3) // buffered channel with capacity 3
Well look at unbuf fered channel s rs t andbuf fered channel s in Sec tion 8.4.4.
8.4.1. Unbuffered Channels
Asendoperat iononanunbuf fered channel blo cks the sendinggoroutine unt i l anot her
goro utine exe cut esacorrespondingreceive onthe samechannel,atwhichpoint the value is
transmitt edand bot h goro utinesmay cont inue. Convers ely, if the receive operat ionwas
attemp ted rs t,the receiving goro utine isblo cke d until another goroutine per for msasendon
thesamechannel.
Communic ationoveranunbuf fered channel causesthe sendingand receiving goro utinesto
sy nchro niz e.Because ofthis, unbuffered channel s aresom etimescal le d sy nchro nou s ch annel s.
Wh enavalue issentonanunbuf fered channel,the receipt ofthe value happ ens before the
re awakeningofthe sendinggoroutine.
In dis cussions ofcon cur rency,whenwesay xhap pens beforey,wedontmeanmerelythat x
occurs earlier in timethan y;wemeanthatitisguarante e d to dosoand thatall its prior
ef fec ts, such as updates tovar iables, arecompleteand thatyou may relyonthem.
Wh en x neit her happens before y norafter y,wesay that xiscon cur rentwit h y.Thisdoesnt
me anthat x and y arenecessarily simultane ous,merelythatwecannot assume anythingabout
their order ing. Aswell see inthe next chapt er, itsnecessary toorder certain eventsdur ingthe
prog ramsexe cut ion toavoid the pro blemsthatarise whentwo goro utinesaccessthe same
var iable conc urrently.
Theclientprogram in Sec tion 8.3 copies inputtothe ser ver in its main goroutine,sothe client
prog ram terminates as soonasthe inp utstreamclos es, even if the backg round goroutine is
st i l l work ing . To makethe program waitfor the backg round goroutine tocompletebefore
exiting , we use a channel tosynchro nize the two goro utines:
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 8.4. CHANNELS 227
gopl.io/ch8/netcat3
func main() {
conn, err := net.Dial("tcp", "localhost:8000")
if err != nil {
log.Fatal(err)
}
done := make(chan struct{})
go func() {
io.Copy(os.Stdout, conn) // NOTE: ignoring errors
log.Println("done")
done <- struct{}{} // signal the main goroutine
}()
mustCopy(conn, os.Stdin)
conn.Close()
<-done // wait for background goroutine to finish
}
Wh enthe userclos es thestand ard inp utstream, mustCopy returnsand the main goroutine
calls conn.Close(),closingbot h halves ofthe networ k connec tion.Closingthe write half of
theconne ction causesthe ser ver tosee anend-of-file condit ion.Closingthe readhalf causes
thebackg round goroutinescal l to io.Copy to retur n a ‘‘re adfro m clos edconne ction’’ er ror,
whichiswhy weve remov edthe error log ging; Exercis e 8.3 sug gests a bettersolut ion.(No tice
that the go st atement cal lsalit eral function, a common con str uction.)
Before itretur ns, the backg round goroutine logs a message , then sends a value onthe done
ch annel.The main goroutine waits until ithas receive d this value beforeretur ning. Asa
resu lt, the program always logs the "done" mess age beforeexiting .
Mess agessentoverchannel s have two imp ortantasp ects. Eachmessage has a value,but
sometimesthe fac t of communic ationand the moment atwhichitocc ursare justas
important. Wecal l mess ages events when wewishtostressthisasp ect.Whenthe event car-
ries noaddition alinfor mat ion, thatis, its solepur pos e is synchro nizat ion, well emp hasize this
by usingachannel whose elementtyp e is struct{},thoug h itscommontouse a channel of
bool or int forthe samepur pos e since done <- 1 is shorter than done <- struct{}{}.
Exercis e 8.3: In netcat3,the int erface value conn hasthe con crete typ e *net.TCPConn,which
repres ents a TCP connec tion.ATCPconne ction con sists oftwo halvesthatmay beclos ed
indep endentlyusingits CloseRead and CloseWrite methods. Modif y themain goroutine of
netcat3 to clos e on lythe write half ofthe conne ction sothatthe program will continueto
pr int the nalech oes fro m the reverb1 server evenafter the stand ard inp uthas beenclos ed.
(D oingthisfor the reverb2 server isharder ; seeExercis e 8.4.)
8.4.2. Pipelines
Channel s canbeusedtoconne ctgoroutinestoget her sothatthe out put of oneisthe inp utto
anot her.Thisiscal le d a pip elin e.The program below con sists ofthree goroutinesconne cte d
by two channel s, as shown sch ematically in Figure8.1.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
228 CHAPTER 8. GOROUTINES AND CHANNELS
Figure 8.1. Athree-stage pip eline.
Thefirs t goro utine, counter,generates the int egers 0,1,2,..., andsends themoverachannel to
thesecon d goro utine, squarer,whichreceiveseachvalue,squ aresit, andsends the resultover
anot her channel tothe thirdgoroutine, pr int er,whichreceivesthe squ are d values andprints
them. For clarity ofthisexample, wehaveint ent ion allychosenver y simplefunctions,thoug h
of cours e they are too computation allytrivialtowar ranttheir own goroutinesinarealist ic
prog ram.
gopl.io/ch8/pipeline1
func main() {
naturals := make(chan int)
squares := make(chan int)
// Counter
go func() {
for x := 0; ; x++ {
naturals <- x
}
}()
// Squarer
go func() {
for {
x:=<-naturals
squares <- x * x
}
}()
// Printer (in main goroutine)
for {
fmt.Println(<-squares)
}
}
As you mig htexp ect,the program printsthe infinite ser ies of squ ares0,1,4,9,and soon.
Pipelines like thismay befound inlon g-r unningser ver programswhere channel s areusedfor
lifelon g communic ationbet weengoroutinescontaininginfinite loops. But what if we wantto
send onlyafinite numberofvalues through the pip eline?
If the sender knows thatnofur thervalues will everbesentonachannel,itisusefultocom-
munic atethisfac t to the receivergoroutinessothatthe y canstopwaiting . Thisisaccom-
plishe d by cl osing thechannel usingthe bui lt-in close func tion:
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 8.4. CHANNELS 229
close(naturals)
Af ter a channel has beenclos ed, any fur thersendoperat ions onitwill panic. After the clos ed
ch annel has been drai ned,thatis, afterthe lastsentelementhas beenreceive d,all subsequent
re ceive operat ions will pro ceed wit houtblo cking but will yield a zerovalue.Closingthe natu-
rals ch annel above wouldcause the squ arerslooptospinasitreceivesanever-endingstream
of zerovalues, andtosendthese zeros tothe print er.
Thereisnoway totestdirec tly whether a channel has beenclos ed, but there isavar iantofthe
re ceive operat ionthatpro duces two results: the receive d ch annel element, plu s aboole an
value,convent ion allycal le d ok,whichis true forasuccessf ulreceive and false forareceive
on a clos edand drained channel.Usingthisfeature , we can modif y thesqu arerslooptostop
when the naturals ch annel isdrained and clos e the squares ch annel intur n.
// Squarer
go func() {
for {
x, ok := <-naturals
if !ok {
break // channel was closed and drained
}
squares <- x * x
}
close(squares)
}()
Becaus e thesyntaxabove isclumsyand thispattern iscommon, the langu agelets ususe a
range lo optoiterateoverchannel s to o.Thisisamoreconvenientsyntaxfor receiving all the
values sentonachannel and ter minat ingthe loopafter the lastone.
In the pip elinebelow,whenthe count ergoroutine finishesits loop after 100 elements, itclos es
the naturals ch annel,causingthe squ arerto finish its loop and clos e the squares ch annel.
(Inamorecomplex program, itmig htmakesense for the count erand squ arerfunctions to
defer the cal lsto close at the outset.) Final ly, the main goroutine finishesits loop and the
prog ram exits.
gopl.io/ch8/pipeline2
func main() {
naturals := make(chan int)
squares := make(chan int)
// Counter
go func() {
for x := 0; x < 100; x++ {
naturals <- x
}
close(naturals)
}()
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
230 CHAPTER 8. GOROUTINES AND CHANNELS
// Squarer
go func() {
for x := range naturals {
squares <- x * x
}
close(squares)
}()
// Printer (in main goroutine)
for x := range squares {
fmt.Println(x)
}
}
Yo u ne e dntclos e ever y ch annel whenyouve finishe d with it. Itsonlynecessary toclos e a
ch annel whenitisimp ortanttotel l thereceiving goro utinesthatall dat a have beensent. A
ch annel thatthe garb agecol lec tor deter mines tobeunreach ablewill haveits res ources
re claimed whether ornot itisclos ed. (Dontconfuse thiswit h theclos e op erat ionfor open
files. It is importanttocal l the Close method onevery file whenyouve finishe d with it.)
At tempt ing toclos e an already-clos edchannel causesapanic, as does closinganil channel.
Closingchannel s hasanother use asabro adc astmechanism, whichwell cov er in Sec tion 8.9.
8.4.3. Unidirectional Channel Types
As programsgrow, itisnatural tobre akuplarge functions int o smal ler pieces. Our pre vious
exampleusedthree goroutines, communic atingovertwo channel s,whichwerelocal variables
of main.The program natural lydiv ides into three functions:
func counter(out chan int)
func squarer(out, in chan int)
func printer(in chan int)
The squarer func tion, sitt ing inthe midd le of the pip eline, takes two parameters, the inp ut
ch annel and the out put channel.Bot h have the sametyp e,but their intended usesare
opp osite: in is onlytobereceive d from,and out is onlytobesentto. The names in and out
conv eythisint ent ion,but still,not hingpre vents squarer from sendingto in or receiving
from out.
Thisarrangement istypic al.Whenachannel issup plie d as a functionparameter,itisnearly
always wit h theint ent thatitbeusedexc lusivelyfor sendingorexc lusivelyfor receiving .
To documentthisint ent and pre ventmisus e,the Gotyp e systemprovides unidirec tional ch an-
nel typ es that expos e on lyone orthe other ofthe sendand receive operat ions.The typ e
chan<- int,asend-only ch annel of int,allowssends but not receives. Convers ely,the typ e
<-chan int,are ceive-only ch annel of int,allowsreceivesbut not sends.(Theposition of the
<- ar row rel ative tothe chan ke yword isamnemonic.) Violation s of thisdis cip lineare
detec ted atcompi letime.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 8.4. CHANNELS 231
Sincethe close op erat ionass ertsthatnomoresends will occ ur on a channel,onlythe send-
inggoroutine isinaposition tocal l it,and for thisreasonitisacompi le-t imeerror toatt emp t
to clos e areceive-onlychannel.
Heresthe squ aring pip elineoncemore, thistimewit h unidirec tion alchannel typ es:
gopl.io/ch8/pipeline3
func counter(out chan<- int) {
for x := 0; x < 100; x++ {
out <- x
}
close(out)
}
func squarer(out chan<- int, in <-chan int) {
for v := range in {
out <- v * v
}
close(out)
}
func printer(in <-chan int) {
for v := range in {
fmt.Println(v)
}
}
func main() {
naturals := make(chan int)
squares := make(chan int)
go counter(naturals)
go squarer(squares, naturals)
printer(squares)
}
Thecal l counter(naturals) implicitlyconverts naturals,a value oftyp e chan int,tothe
type ofthe parameter, chan<- int.The printer(squares) call doesasimi lar implicitcon-
versionto <-chan int.Conversions fro m bidirec tion altounidirec tion alchannel typ es are
permit ted inany assig nment. Thereisnogoing back, how ever: onceyou haveavalue ofa
unidirec tion altyp e such as chan<- int,there isnoway toobt ain fro m it a value oftyp e
chan int that referstothe samechannel dat a st ruc ture.
8.4.4. Buffered Channels
Abuf fered channel has a queue ofelements. Thequeuesmaximum size isdeter mined whenit
is cre ate d, by the cap acity argumentto make.The statement below cre atesabuf fered channel
capableofholdingthree string values. Figure8.2 isagraphic al repres entation of ch andthe
ch annel towhichitrefers.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
232 CHAPTER 8. GOROUTINES AND CHANNELS
ch = make(chan string, 3)
Figure 8.2. An emp tybuf fered channel.
Asendoperat iononabuf fered channel ins ertsanelementatthe backofthe queue,and a
re ceive operat ionremov es an elementfro m thefro nt. Ifthe channel isfull,the sendoperat ion
blocks its goroutine unt i l sp ace ismade availablebyanother goroutinesreceive . Conv ers ely,if
thechannel isemp ty, a receive operat ionblo cks until a value issentbyanother goroutine.
We can senduptothree values onthischannel wit houtthe goroutine blo cking:
ch <- "A"
ch <- "B"
ch <- "C"
At thispoint,the channel isfull (Figure8.3), andafourthsendstatement wou ldblo ck.
Figure 8.3. Afull buf fered channel.
If wereceive one value,
fmt.Println(<-ch) // "A"
thechannel isneither full nor emp ty(Figure8.4), soeit her a sendoperat ionorareceive oper-
at ioncou ldpro ceed wit houtblo cking . In thisway,the channelsbuf fer decoup les the sending
andreceiving goro utines.
Figure 8.4. Apar tiallyfull buf fered channel.
In the unlikelyevent thataprogram needstoknowthe channelsbuf fer cap acity,itcan be
obtained bycal lingthe bui lt-in cap func tion:
fmt.Println(cap(ch)) // "3"
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 8.4. CHANNELS 233
Wh enapp lie d to a channel,the bui lt-in len func tionretur nsthe numberofelements cur-
rent lybuf fered.Since in a conc urrentprogram thisinfor mat ionislikelytobestale as soonas
it isret rie ved,its value islimite d, butitcou ldcon ceivably beusefuldur ingfau ltdiagnosisor
perfor mance opt imizat ion.
fmt.Println(len(ch)) // "2"
Af ter two morereceive operat ions the channel isemp tyagain, andafourthwou ldblo ck:
fmt.Println(<-ch) // "B"
fmt.Println(<-ch) // "C"
In thisexample, the sendand receive operat ions were all per for med bythe samegoroutine,
butinrealprogramsthe y areusu allyexe cut edbydif ferentgoroutines. Nov ices aresom etimes
temp ted touse buf fered channel s within a singlegoroutine asaqueue,lured bytheir ple as-
inglysimplesyntax, but thisisamistake . Channel s aredeeply conne cte d to goroutine
sche duling, and wit houtanother goroutine receiving fro m thechannel,asenderandperhaps
thewhole programrisksbecomingblo cke d fore ver.Ifall you need isasimplequeue,make
on e usingaslice.
Theexamplebelow shows an applic ationofabuf fered channel.Itmakes paral lelrequests to
thre e mi rro rs,thatis, equivalentbut geographic ally distr ibute d servers. Itsends their
resp ons esoverabuf fered channel,thenreceivesand retur nsonlythe rs t resp ons e,whichis
thequickest one toarrive . Thus mirroredQuery returnsaresultevenbeforethe two slower
servershaveresponded.(In cidentally, itsquite nor mal for several goroutinestosendvalues to
thesamechannel con cur rently, as in thisexample, ortoreceive fro m thesamechannel.)
func mirroredQuery() string {
responses := make(chan string, 3)
go func() { responses <- request("asia.gopl.io") }()
go func() { responses <- request("europe.gopl.io") }()
go func() { responses <- request("americas.gopl.io") }()
return <-responses // return the quickest response
}
func request(hostname string) (response string) { /* ... */ }
Hadweusedanunbuf fered channel,the two slowergoroutineswou ldhavegot ten stuck trying
to sendtheir respons esonachannel fro m whichnogoroutine will everreceive . Thissit-
uation,cal le d a goro utine leak,wou ldbeabug . Un likegarb agevar iables, leaked goroutinesare
notaut omat ical lycol lec ted,soitisimp ortanttomakesurethatgoroutinester minatethem-
selves whennolon g erneeded.
Thechoice bet weenunbuf fered and buf fered channel s,and the choice ofabuf fered channels
capacity,may bot h af fec t thecor rec tnessofaprogram. Unbuf fered channel s give stron g er
sy nchro nizat ionguarante esbecause ever y send operat ionissynchro nize d with its cor-
resp ondingreceive;wit h buffered channel s,these operat ions are decoupled.Als o,whenwe
know anupp erbound onthe numberofvalues thatwill besentonachannel,itsnot unu sual
to cre ate a buf fered channel ofthatsize andper for m al l thesends beforethe rs t value is
re ceive d.Fai luretoallocatesufficientbuf fer cap acity wou ldcause the program todeadlock.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
234 CHAPTER 8. GOROUTINES AND CHANNELS
Channel buf fer ingmay also affec t prog ram perfor mance.Imagine three cooks in a cakeshop,
on e baking , on e icing, and one ins cribing eachcakebeforepassingitontothe next cook inthe
assemb l y line. Inakit c hen wit h litt lespace,eachcookthathas finishe d acakemustwaitfor
thenext cook tobecom e re ady toacceptit; thisrende zvou s is analogou s to communic ation
ov er an unbuf fered channel.
If there isspace for onecakebet weeneachcook, a cook may place a finishe d cake there and
immediate lystart wor k on the next; thisisanalogou s to a buf fered channel wit h capacity 1.So
long asthe cooks wor k at aboutthe samerateonaverage, mostofthese handovers pro ceed
quickly, smo othingout transientdif ferences in their respectiverates. Morespace bet ween
co okslargerbuf ferscansmo oth out bigger transientvar iat ions intheir rates wit hout
st allingthe ass emb l y line, suchashappens whenone cooktakes a short bre ak, thenlater
rushestocatch up.
On the other hand, ifanearlier stage ofthe ass emb l y lineiscon sistent lyfasterthanthe fol low-
ingstage , thebuf fer bet weenthemwill spend most ofits timefull.Convers ely,ifthe later
st age isfaster, the buf fer will usu allybeemp ty. A buffer provides nobenefit in thiscas e.
Theass emb l y linemet aphorisausefulone for channel s andgoroutines. For example, ifthe
second stage ismoreelaborate, a singlecookmay not beabletokeepupwit h thesup ply fro m
thefirs t co okormeetthe demandfro m thethird. Tosolvethe pro blem, wecou ldhireanother
co oktohelpthe secon d,per for mingthe sametaskbut wor kingindep endently. Thisisanalo-
gous tocre ating another goroutine communic atingoverthe samechannel s.
We donthavespace toshowithere, but the gopl.io/ch8/cake packagesimulates thiscake
shop, wit h several parametersyou can vary. Itincludes benchmarks(§11.4) for a few ofthe
scenar ios des crib edabove .
8.5. Looping in Parallel
In thissec tion,well explore som e common con cur rency patternsfor exe cut ing all the itera-
tion s of a loopinparal lel. Well con sider the pro blem ofpro ducingthumbnail-size images
from a set offull-size ones. The gopl.io/ch8/thumbnail packageprovides an ImageFile
func tionthatcan scale a singleimage . We wontshowits implementation but it can be
down loade d from gopl.io.
gopl.io/ch8/thumbnail
package thumbnail
// ImageFile reads an image from infile and writes
// a thumbnail-size version of it in the same directory.
// It returns the generated file name, e.g., "foo.thumb.jpg".
func ImageFile(infile string) (string, error)
Theprogram below loops overalistofimage file names andpro duces a thumbnail for each
on e:
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 8.5. LOOPING IN PARALLEL 235
gopl.io/ch8/thumbnail
// makeThumbnails makes thumbnails of the specified files.
func makeThumbnails(filenames []string) {
for _, f := range filenames {
if _, err := thumbnail.ImageFile(f); err != nil {
log.Println(err)
}
}
}
Obviou sly the order in whichwepro cessthe files doesntmatter, since eachscalingoperat ion
is indep endentofall the others. Pro blemslikethisthatcon sistent ire lyofsubproblemsthat
arecompletelyindep endentofeachother aredes crib edas embarrassingly paral lel.Emb arrass-
inglyparal lelpro blemsare the easiestkindtoimp lementcon cur rentlyand enj oyper for mance
that scales linearlywit h theamountofparal lelism.
Letsexe cut e al l thes e op erat ions inparal lel, there byhidingthe latency ofthe file I/O and
usingmultipleCPUsfor the image-s calingcomputation s.Our rs t attemp t at a conc urrent
versionjustaddsago ke yword . Well ignoreerror s fornow and addressthemlater.
// NOTE: incorrect!
func makeThumbnails2(filenames []string) {
for _, f := range filenames {
go thumbnail.ImageFile(f) // NOTE: ignoring errors
}
}
Thisversionrunsreallyfastto o fast, in fact, since ittakes lesstimethanthe original, even
when the slice of file names cont ainsonlyasingleelement. Iftheresnoparal lelism, how can
thecon cur rentversionpossibly run faster? Theansweristhat makeThumbnails returns
before ithas finishe d doingwhatitwas sup pos edtodo. Itstartsall the goroutines, one per file
name,but doesntwaitfor themto finish.
Thereisnodirec t way towaitunt i l agoroutine has finishe d,but wecan change the inner
goro utine torep ort its comp let iontothe out ergoroutine bysendinganevent onashare d
ch annel.Since weknowthatthere are exac tly len(filenames) inner goroutines, the out er
goro utine need onlycount thatmanyeventsbeforeitretur ns:
// makeThumbnails3 makes thumbnails of the specified files in parallel.
func makeThumbnails3(filenames []string) {
ch := make(chan struct{})
for _, f := range filenames {
go func(f string) {
thumbnail.ImageFile(f) // NOTE: ignoring errors
ch <- struct{}{}
}(f)
}
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
236 CHAPTER 8. GOROUTINES AND CHANNELS
// Wait for goroutines to complete.
for range filenames {
<-ch
}
}
No tice thatwepassedthe value of f as an explicitargumenttothe lit eral functionins teadof
usingthe declarat ionof f from the enclosing for lo op:
for _, f := range filenames {
go func() {
thumbnail.ImageFile(f) // NOTE: incorrect!
// ...
}()
}
Re call the pro blem ofloopvar iable capture inside an anonymou s func tion, descr ibedin
Section5.6.1. Above , thesinglevar iable f is share d by all the anony mou s func tionvalues and
up dated bysuccessive loopiterat ions.Bythe timethe newgoroutinesstart exe cut ing the lit-
eral function, the for lo opmay haveupdated f andstarted another iterat ionor(more likely)
finishe d entire ly, sowhenthese goroutinesreadthe value of f,the y al l observeittohavethe
value ofthe nalelementofthe slice.Byaddinganexplicitparameter,weens ure thatweuse
thevalue of f that iscur rentwhenthe go st atement isexe cut ed.
Wh atifwewanttoretur n values fro m each wor ker goroutine tothe main one?Ifthe cal l to
thumbnail.ImageFile fai lstocre ate a file,itretur ns an error.The next versionof
makeThumbnails returnsthe rs t er ror it receivesfro m anyofthe scalingoperat ions:
// makeThumbnails4 makes thumbnails for the specified files in parallel.
// It returns an error if any step failed.
func makeThumbnails4(filenames []string) error {
errors := make(chan error)
for _, f := range filenames {
go func(f string) {
_, err := thumbnail.ImageFile(f)
errors <- err
}(f)
}
for range filenames {
if err := <-errors; err != nil {
return err // NOTE: incorrect: goroutine leak!
}
}
return nil
}
Thisfunctionhas a subtlebug . Wh enitencount ers the rs t non-ni l er ror,itretur nsthe error
to the cal ler,leaving nogoroutine drainingthe errors ch annel.Eachremainingwor ker
goro utine will blo ckforever whenittries tosendavalue onthatchannel,and will never
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 8.5. LOOPING IN PARALLEL 237
terminate. Thissit uat ion, a goroutine leak(§8.4.4), may cause the whole program toget stuck
or torun out of memor y.
Thesimplestsolut ion istouse a buf fered channel wit h sufficientcap acity thatnowor ker
goro utine will blo ckwhenitsends a message . (Analt ernat ive solut ion istocre ate another
goro utine todrain the channel whi le themain goroutine retur nsthe rs t er ror wit houtdel ay.)
Thenext versionof makeThumbnails us esabuf fered channel toretur n thenames ofthe gen-
erated image files along wit h anyerror s.
// makeThumbnails5 makes thumbnails for the specified files in parallel.
// It returns the generated file names in an arbitrary order,
// or an error if any step failed.
func makeThumbnails5(filenames []string) (thumbfiles []string, err error) {
type item struct {
thumbfile string
err error
}
ch := make(chan item, len(filenames))
for _, f := range filenames {
go func(f string) {
var it item
it.thumbfile, it.err = thumbnail.ImageFile(f)
ch <- it
}(f)
}
for range filenames {
it := <-ch
if it.err != nil {
return nil, it.err
}
thumbfiles = append(thumbfiles, it.thumbfile)
}
return thumbfiles, nil
}
Ourfinalversionof makeThumbnails,below,retur nsthe tot alnumberofbytes occ upied by
thenew files. Unlikethe pre vious versions,how ever, itreceivesthe file names not asaslice but
ov erachannel ofstr ings, sowecannot pre dic t thenumberofloopiterat ions.
To k nowwhenthe lastgoroutine has finishe d (w hichmay not bethe lastone tostart), weneed
to increment a count erbeforeeachgoroutine startsand decrement itaseachgoroutine fin-
ishes. Thisdemands a speci al kind ofcount er, one thatcan besafelymanipu lated fro m
mu ltiplegoroutinesand thatprovides a way towaitunt i l it becom eszero. Thiscount ertyp e is
know n as sync.WaitGroup,and the codebelow shows how touse it:
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
238 CHAPTER 8. GOROUTINES AND CHANNELS
// makeThumbnails6 makes thumbnails for each file received from the channel.
// It returns the number of bytes occupied by the files it creates.
func makeThumbnails6(filenames <-chan string) int64 {
sizes := make(chan int64)
var wg sync.WaitGroup // number of working goroutines
for f := range filenames {
wg.Add(1)
// worker
go func(f string) {
defer wg.Done()
thumb, err := thumbnail.ImageFile(f)
if err != nil {
log.Println(err)
return
}
info, _ := os.Stat(thumb) // OK to ignore error
sizes <- info.Size()
}(f)
}
// closer
go func() {
wg.Wait()
close(sizes)
}()
var total int64
for size := range sizes {
total += size
}
return total
}
No tethe asy mmet ryinthe Add and Done methods. Add,whichincrementsthe count er, must
be cal le d before the wor ker goroutine starts, not wit hin it; other wis e we wouldnot besurethat
the Add happ ens before the ‘‘clos er’’ goro utine cal ls Wait.Als o, Add takesaparameter,but Done
do es not; itsequivalentto Add(-1).Weuse defer to ens ure thatthe count erisdecrement ed
even in the error cas e.The str uctureofthe codeabove isacommonand idiomaticpattern for
lo oping inparal lelwhenwedontknowthe numberofiterat ions.
The sizes ch annel car r ies each file size backtothe main goroutine,whichreceivesthem
usingarange lo opand computesthe sum. Obs erve how wecre ate a clos er goro utine that
waits for the wor kersto finish beforeclosingthe sizes ch annel.These two operat ions,wait
andclos e,mustbecon cur rentwit h theloopover sizes.Con sider the alt ernat ives: if the wait
op erat ionwereplace d in the main goroutine beforethe loop, itwou ldnever end, and ifplace d
af ter the loop, itwou ldbeunreach ablesince wit h nothingclosingthe channel,the loopwou ld
ne ver ter minate.
Figure8.5 illustrates the sequence ofeventsinthe makeThumbnails6 func tion. Thever tic al
lines represent goroutines. Thethin seg ments indic atesle ep,the thickseg ments activity.The
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 8.6. EXAMPLE: CONCURRENT WEB CRAWLER 239
Figure 8.5. Thesequence ofeventsin makeThumbnails6.
di agon alarrowsindic ateeventsthatsynchro nize one goroutine wit h anot her.Time ows
down.Not ice how the main goroutine spend s most ofits timeinthe range lo opasleep, wait-
ingfor a wor ker tosendavalue orthe clos er to clos e thechannel.
Exercis e 8.4: Mo dif y the reverb2 server touse a sync.WaitGroup perconne ction tocount
thenumberofactive echo goro utines. Whenitfal lstozero, clos e thewrite half ofthe TCP
connec tion asdes crib edinExercis e 8.3. Ver ifythatyourmodified netcat3 clientfro m that
exercise waits for the nalech oes ofmultiplecon cur rentshouts, evenafter the stand ard inp ut
hasbeenclos ed.
Exercis e 8.5: Take anexist ing CPU-b oundsequential program, suchasthe Mandelbro t
prog ram of Sec tion 3.3 orthe 3-D sur face comp utation of Sec tion 3.2, andexe cut e itsmain
lo op in paral lelusingchannel s forcommunic ation. How muchfasterdoesitrun ona
mu ltiprocessormachine? Whatisthe opt imalnumberofgoroutinestouse?
8.6. Example: ConcurrentWeb Crawler
In Sec tion 5.6, wemade a simpleweb craw ler that explore d thelin k graphofthe web in
breadt h-firs t order.Inthissec tion,we ll makeitcon cur rentsothatindep endentcal lsto crawl
canexploit the I/O paral lelism availableinthe web.The crawl func tionremainsexac tly asit
was in gopl.io/ch5/findlinks3:
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
240 CHAPTER 8. GOROUTINES AND CHANNELS
gopl.io/ch8/crawl1
func crawl(url string) []string {
fmt.Println(url)
list, err := links.Extract(url)
if err != nil {
log.Print(err)
}
return list
}
Themain functionres embles breadthFirst (§5.6). Asbefore, a wor klist records the queue of
it ems thatneed pro cessing, eachitembeingalistofURLs tocraw l,but thistime, ins teadof
repres enting the queue usingaslice,weuse a channel.Eachcal l to crawl occurs inits own
goro utine and sends the lin ks it dis cov ers backtothe wor klist.
func main() {
worklist := make(chan []string)
// Start with the command-line arguments.
go func() { worklist <- os.Args[1:] }()
// Crawl the web concurrently.
seen := make(map[string]bool)
for list := range worklist {
for _, link := range list {
if !seen[link] {
seen[link] = true
go func(link string) {
worklist <- crawl(link)
}(link)
}
}
}
}
No tice thatthe craw l goro utine takes link as an explicitparameter toavoid the pro blem of
lo opvar iable capture wefirs t saw in Sec tion 5.6.1. Als o notice thatthe initial sendofthe com-
mand-linearguments tothe wor klist mustrun in its own goroutine toavoid dead lock,astuck
situ ation inwhichbot h themain goroutine and a craw ler goro utine att emp t to sendtoeach
ot her whi le neit her isreceiving . An alt ernat ive solut ion wou ldbetouse a buf fered channel.
Thecraw ler is now hig hly con cur rentand printsastorm ofURLs, but it has two pro blems.
The rs t problem manifests its elf as erro r mess ages in the log afterafew second s of operat ion:
$gobuild gopl.io/ch8/crawl1
$./crawl1 http://gopl.io/
http://gopl.io/
https://golang.org/help/
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 8.6. EXAMPLE: CONCURRENT WEB CRAWLER 241
https://golang.org/doc/
https://golang.org/blog/
...
2015/07/15 18:22:12 Get ...: dial tcp: lookup blog.golang.org: no such host
2015/07/15 18:22:12 Get ...: dial tcp 23.21.222.120:443: socket:
too many open files
...
Theinitial erro r mess age isasur prisingrep ort ofaDNS lookup fai lurefor a reliabledom ain.
Thesubsequenterror message reveals the cause:the program create d so manynet wor k con-
ne ction s at oncethatitexceeded the per-pro cesslimitonthe numberofopenfiles, causing
op erat ions such as DNS lookupsand cal lsto net.Dial to start fai ling.
Theprogram is to o paral lel. Unb ounde d paral lelism israrelyagood ide a since there isalways
alimiting fac tor inthe system, suchasthe numberofCPU coresfor compute-b ound
work loads,the numberofspind les andheads for local disk I/O operat ions,the bandw idt h of
thenet wor k forstreamingdow nlo ads, orthe ser vingcap acity ofa web ser vice.The solut ion is
to limitthe numberofparal lelusesofthe res ource tomatch the level ofparal lelism thatis
avai lable.Asimpleway todothatinour exampleistoens ure thatnomorethan n callsto
links.Extract areactiveatonce, where n is comfortably lessthanthe file descr ipt or
limit20, say.Thisisanalogou s to the way a door man at a crow ded nig htclub admits a guest
on lywhensom e ot her guestleaves.
We can limitparal lelism usingabuf fered channel ofcap acity n to model a con cur rency primi-
tive cal le d a counting semap hore.Con ceptu ally, eachofthe n vac antslots in the channel buf fer
repres ents a token entitlingthe holder topro ceed.Sendingavalue into the channel acquires a
token, andreceiving a value fro m thechannel releasesatoken, creating a new vac antslot.
Thisens uresthatatmost n send s canocc ur withoutanint erveningreceive . (A lthough it
mig htbemoreint uit ive totre at ll ed slotsinthe channel buf fer as tokens, usingvac antslots
avoids the need tofill the channel buf fer aftercre ating it.) Since the channel elementtyp e is
notimp ortant, well use struct{},whichhas size zero.
Letsrewrite the crawl func tionsothatthe cal l to links.Extract is brackete d by operat ions
to acquireand release a token, thu s ensuring thatatmost20cal lstoitare activeatone time.
Itsgood prac tice tokeepthe semaphore operat ions asclos e as possibletothe I/O operat ion
they regu late.
gopl.io/ch8/crawl2
// tokens is a counting semaphore used to
// enforce a limit of 20 concurrent requests.
var tokens = make(chan struct{}, 20)
func crawl(url string) []string {
fmt.Println(url)
tokens <- struct{}{} // acquire a token
list, err := links.Extract(url)
<-tokens // release the token
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
242 CHAPTER 8. GOROUTINES AND CHANNELS
if err != nil {
log.Print(err)
}
return list
}
Thesecon d problem isthatthe program never ter minates, evenwhenithas discov ere d al l the
lin ks re ach ablefro m theinitial URLs. (Of course,youre unlikelytonot ice thispro blem unless
youcho ose the initial URLs caref ullyorimp lementthe depth-limiting feature ofExercis e 8.6.)
Fo r theprogram toter minate, weneed tobre akout of the main loop whenthe wor klist is
empt y an d no craw l goro utinesare active.
func main() {
worklist := make(chan []string)
var n int // number of pending sends to worklist
// Start with the command-line arguments.
n++
go func() { worklist <- os.Args[1:] }()
// Crawl the web concurrently.
seen := make(map[string]bool)
for ; n > 0; n-- {
list := <-worklist
for _, link := range list {
if !seen[link] {
seen[link] = true
n++
go func(link string) {
worklist <- crawl(link)
}(link)
}
}
}
}
In thisversion, the count er n ke eps trackofthe numberofsends tothe wor klist thatare yet to
occur. Eachtimeweknowthatanitemneedstobesenttothe wor klist,weincrement n,once
before wesendthe initial command-linearguments, andagain eachtimewestart a craw ler
goro utine.The main loop ter minates when n fal lstozero, since there isnomorewor k to be
done.
No w thecon cur rentcraw ler runs about 20times fasterthanthe bre adt h-firs t craw ler from
Section5.6, wit houterror s,and ter minates correc tly ifitshouldcompleteits task.
Theprogram below shows an alternat ive solut ion tothe pro blem ofexcessive con cur rency.
Thisversionusesthe original crawl func tionthathas nocount ing semaphore , butcal lsit
from oneof20lon g-live d craw ler goro utines, thu s ensuring thatatmost 20 HTTPrequests are
ac tivecon cur rently.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 8.6. EXAMPLE: CONCURRENT WEB CRAWLER 243
gopl.io/ch8/crawl3
func main() {
worklist := make(chan []string) // lists of URLs, may have duplicates
unseenLinks := make(chan string) // de-duplicated URLs
// Add command-line arguments to worklist.
go func() { worklist <- os.Args[1:] }()
// Create 20 crawler goroutines to fetch each unseen link.
for i := 0; i < 20; i++ {
go func() {
for link := range unseenLinks {
foundLinks := crawl(link)
go func() { worklist <- foundLinks }()
}
}()
}
// The main goroutine de-duplicates worklist items
// and sends the unseen ones to the crawlers.
seen := make(map[string]bool)
for list := range worklist {
for _, link := range list {
if !seen[link] {
seen[link] = true
unseenLinks <- link
}
}
}
}
Thecraw ler goro utinesare all fed bythe samechannel, unseenLinks.The main goroutine is
resp onsible for de-dup lic atingitems itreceivesfro m thewor klist,and thensendingeach
unseen one overthe unseenLinks ch annel toacraw ler goro utine.
The seen mapis conne d within the main goroutine;thatis, itcan beaccessedonlybythat
goro utine.Likeother for msofinfor mat ionhiding, confinement helps usreasonabout the
correc tnessofaprogram. For example, local variables cannot bemention edbynamefro m
outside the functioninwhichthe y aredeclare d;var iables thatdonot escap e (§2.3.4) fro m a
func tioncannot beaccessedfro m outside thatfunction; andenc apsulated elds ofanobj e ct
cannot beaccessedexceptbythe methodsofthatobj e ct. Inall cas es, infor mat ionhidinghelps
to limitunintended int erac tions bet weenpar ts of the program.
Linksfound by crawl aresenttothe wor klist fro m adedic ated goroutine toavoid dead lock.
To s avespace,wehavenot addressedthe pro blem ofter minat ion in thisexample.
Exercis e 8.6: Adddepth-limiting tothe con cur rentcraw ler.Thatis, if the usersets -depth=3,
then onlyURLs reach able by at mostthree lin ks wi l l be fetch ed.
Exercis e 8.7: Wr ite a con cur rentprogram thatcre atesalocal mirro r of a web sit e , fetching
each reach ablepageand writing ittoadirec tor y on the local disk.Onlypages wit hin the
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
244 CHAPTER 8. GOROUTINES AND CHANNELS
or iginaldom ain (for ins tance, golang.org)shouldbefetch ed. URLs wit hin mirro redpages
shouldbealt ere d as needed sothatthe y refertothe mir rored page , notthe original.
8.7. Multiplexing with select
Theprogram below doesthe count dow n forarocketlaunch.The time.Tick func tionretur ns
achannel onwhichitsends eventsper iodic ally,actinglikea met ron om e.The value ofeach
eventisatimestamp,but it israrely as int erest ing asthe fac t of itsdeliver y.
gopl.io/ch8/countdown1
func main() {
fmt.Println("Commencing countdown.")
tick := time.Tick(1 * time.Second)
for countdown := 10; countdown > 0; countdown-- {
fmt.Println(countdown)
<-tick
}
launch()
}
No w letsadd the abi lit y to abort the launch sequence bypressingthe retur n ke y during the
countdow n.First,westart a goroutine thattries toreadasinglebytefro m thestand ard inp ut
and, ifitsucce e ds, sends a value onachannel cal le d abort.
gopl.io/ch8/countdown2
abort := make(chan struct{})
go func() {
os.Stdin.Read(make([]byte, 1)) // read a single byte
abort <- struct{}{}
}()
No w each iterat ionofthe count dow n lo opneedstowaitfor anevent toarrive onone ofthe
twochannel s:the tickerchannel ifeverythingisfine (‘‘nominal’’ in NASAjargon) oranabort
eventifthere was an ‘‘anom aly.’’ We cantjustreceive fro m each channel because whiche ver
op erat ionwetry rs t wi l l blockunt i l comp let ion. Weneed to mu ltipl ex thes e op erat ions,and
to dothat, weneed a select statement.
select {
case <-ch1:
// ...
case x := <-ch2:
// ...use x...
case ch3 <- y:
// ...
default:
// ...
}
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 8.7. MULTIPLEXING WITH SELECT 245
Thegeneral for m of a selec t st atement isshown above . Like a switch statement,ithas a num-
berofcas es andanopt ion al default.Eachcas e sp ecifies a commu nicati on (a sendorreceive
op erat iononsom e ch annel)and anass oci ated blo ckofstatements. A re ceive expressionmay
appear onits own,asinthe rs t case,orwit hin a short var iable decl arat ion, as in the secon d
case;the secon d form lets you refer tothe receive d value.
A select waits until a communic ationfor som e case isready topro ceed.Itthenper for ms
that communic ationand exe cut esthe cas esass oci ated statements; the other communic ations
do not happen. A select with nocas es, select{},waits forever.
Letsretur n to our rocketlaunch program. The time.After func tionimmediate lyretur nsa
ch annel,and startsanew goroutine thatsends a singlevalue onthatchannel after the speci-
ed time.The selec t st atement below waits until the rs t of two eventsarrives, either an abor t
eventorthe event indic atingthat10secon dshaveelaps ed. If10secon dsgobywit h no abort,
thelaunch pro ceeds.
func main() {
// ...create abort channel...
fmt.Println("Commencing countdown. Press return to abort.")
select {
case <-time.After(10 * time.Second):
// Do nothing.
case <-abort:
fmt.Println("Launch aborted!")
return
}
launch()
}
Theexamplebelow ismoresubtle. The channel ch,whose buf fer size is1,isalt ernatelyemp ty
then full,soonlyone ofthe cas es canpro ceed,eit her the sendwhen i is even, orthe receive
when i is odd.Italways prints 02468.
ch := make(chan int, 1)
for i := 0; i < 10; i++ {
select {
case x := <-ch:
fmt.Println(x) // "0" "2" "4" "6" "8"
case ch <- i:
}
}
If multiplecas es areready, select picksone atrandom, whichens uresthatevery channel has
an equ alchanceofbeingselec ted.Incre asingthe buf fer size ofthe pre vious examplemakes its
output non deter minist ic, because whenthe buf fer isneither full nor emp ty, the selec t st ate-
ment figurat ive lytossesacoin.
Letsmakeour launch program print the count dow n.The selec t st atement below causeseach
it erat ionofthe looptowaitupto1secon d for an abort,but nolon g er.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
246 CHAPTER 8. GOROUTINES AND CHANNELS
gopl.io/ch8/countdown3
func main() {
// ...create abort channel...
fmt.Println("Commencing countdown. Press return to abort.")
tick := time.Tick(1 * time.Second)
for countdown := 10; countdown > 0; countdown-- {
fmt.Println(countdown)
select {
case <-tick:
// Do nothing.
case <-abort:
fmt.Println("Launch aborted!")
return
}
}
launch()
}
The time.Tick func tionbeh avesasifitcre atesagoroutine thatcal ls time.Sleep in a loop,
sendinganevent eachtimeitwakes up. Whenthe count dow n func tionabove retur ns, itstops
re ceiving eventsfro m tick,but the tickergoroutine isstill there , tr yinginvain tosendona
ch annel fro m whichnogoroutine isreceivinga goro utine leak (§8.4.4).
The Tick func tionisconvenient, but itsappro priateonlywhenthe ticks will beneeded
thro ughoutthe lifet imeofthe app lic ation. Other wis e,weshoulduse thispattern:
ticker := time.NewTicker(1 * time.Second)
<-ticker.C // receive from the ticker'schannel
ticker.Stop() // cause the ticker'sgoroutine to terminate
Sometimeswewanttotry tosendorreceive onachannel but avoid blo cking ifthe channel is
notreadya non-blocki n g communic ation. A sele ctstatement can do thattoo.Aselect
mayhaveadefault,whichspecifies whattodowhennon e of the other communic ations can
proceed immediate ly.
Theselec t st atement below receivesavalue fro m the abort ch annel ifthere isone toreceive;
ot her wis e it doesnot hing. Thisisanon-blo cking receive operat ion; doingitrep eatedlyis
called polling achannel.
select {
case <-abort:
fmt.Printf("Launch aborted!\n")
return
default:
// do nothing
}
Thezerovalue for a channel is nil.Perhaps sur prisingly, nil channel s aresom etimesuseful.
Becaus e send and receive operat ions onanil channel blo ckforever,acas e in a selec t st atement
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 8.8. EXAMPLE: CONCURRENT DIRECTORYTRAVERSAL 247
whos e ch annel isnil isnever selec ted.Thislets ususe nil to enableordis ablecas es that cor-
resp ond tofeatureslikehandlingtimeoutsorcancellat ion, respondingtoother inputevents,
or emitt ing out put.Well see anexample in the next sec tion.
Exercis e 8.8: Usingaselec t st atement,add a timeout tothe ech o server fro m Section8.3 so
that itdis conne cts anyclientthatshoutsnot hingwit hin 10 secon ds.
8.8. Example: ConcurrentDirectory Traversal
In thissec tion,well bui ld aprogram thatrep ortsthe diskusage ofone ormoredirec tor ies
sp ecified onthe command line, likethe Unix du command. Mostofits wor k is don e by the
walkDir func tionbelow,whichenumerates the ent ries ofthe direc tor y dir usingthe dirents
helperfunction.
gopl.io/ch8/du1
// walkDir recursively walks the file tree rooted at dir
// and sends the size of each found file on fileSizes.
func walkDir(dir string, fileSizes chan<- int64) {
for _, entry := range dirents(dir) {
if entry.IsDir() {
subdir := filepath.Join(dir, entry.Name())
walkDir(subdir, fileSizes)
}else {
fileSizes <- entry.Size()
}
}
}
// dirents returns the entries of directory dir.
func dirents(dir string) []os.FileInfo {
entries, err := ioutil.ReadDir(dir)
if err != nil {
fmt.Fprintf(os.Stderr, "du1: %v\n", err)
return nil
}
return entries
}
The ioutil.ReadDir func tionretur nsa slice of os.FileInfothesameinfor mat ionthata
call to os.Stat returnsfor a single file.For eachsub direc tor y, walkDir re cursive lycal lsits elf,
andfor each file, walkDir send s amessage onthe fileSizes ch annel.The mess age isthe size
of the file in bytes.
Themain function, shown below,usestwo goro utines. Thebackg round goroutine cal ls
walkDir foreachdirec tor y sp ecified onthe command lineand nallyclos es the fileSizes
ch annel.The main goroutine computesthe sum ofthe file sizes itreceivesfro m thechannel
and nallyprintsthe tot al.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
248 CHAPTER 8. GOROUTINES AND CHANNELS
// The du1 command computes the disk usage of the files in a directory.
package main
import (
"flag"
"fmt"
"io/ioutil"
"os"
"path/filepath"
)
func main() {
// Determine the initial directories.
flag.Parse()
roots := flag.Args()
if len(roots) == 0 {
roots = []string{"."}
}
// Traverse the file tree.
fileSizes := make(chan int64)
go func() {
for _, root := range roots {
walkDir(root, fileSizes)
}
close(fileSizes)
}()
// Print the results.
var nfiles, nbytes int64
for size := range fileSizes {
nfiles++
nbytes += size
}
printDiskUsage(nfiles, nbytes)
}
func printDiskUsage(nfiles, nbytes int64) {
fmt.Printf("%d files %.1f GB\n", nfiles, float64(nbytes)/1e9)
}
Thisprogram pausesfor a lon g whilebeforeprint ing its result:
$gobuild gopl.io/ch8/du1
$./du1 $HOME /usr /bin /etc
213201 files 62.7 GB
Theprogram wou ldbenicer if itkeptusinfor med ofits progress. How ever, simply mov ing
the printDiskUsage call int o theloopwou ldcause ittoprint thous and s of lines ofout put.
Thevar iantof du belowprintsthe tot als per iodic ally,but only ifthe -v flag isspecified since
notall users will wanttosee progressmessages. Thebackg round goroutine thatloops over
roots remainsunchange d.The main goroutine nowusesatickertogenerateeventsevery
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 8.8. EXAMPLE: CONCURRENT DIRECTORYTRAVERSAL 249
500ms, andaselec t st atement towaitfor eit her a file size message , in whichcas e it updatesthe
totals,oratickevent,inwhichcas e it prints the cur renttot als.Ifthe -v flag isnot specified,
the tick ch annel remainsnil,and its cas e in the select is effec tive lydis abled.
gopl.io/ch8/du2
var verbose = flag.Bool("v", false, "show verbose progress messages")
func main() {
// ...start background goroutine...
// Print the results periodically.
var tick <-chan time.Time
if *verbose {
tick = time.Tick(500 * time.Millisecond)
}
var nfiles, nbytes int64
loop:
for {
select {
case size, ok := <-fileSizes:
if !ok {
break loop // fileSizes was closed
}
nfiles++
nbytes += size
case <-tick:
printDiskUsage(nfiles, nbytes)
}
}
printDiskUsage(nfiles, nbytes) // final totals
}
Sincethe program nolon g erusesarange lo op, the rs t select case mustexp
licitlytest
whet her the fileSizes ch annel has beenclos ed, usingthe two-resultfor m of receive opera-
tion.Ifthe channel has beenclos ed, the program bre aks out of the loop. The lab ele d break
st atement bre aks out of bot h the select andthe for lo op; an unlabele d break wouldbre ak
outofonlythe select,causingthe looptobeg in thenext iterat ion.
Theprogram now gives usaleisure lystreamofupdates:
$gobuild gopl.io/ch8/du2
$./du2 -v $HOME /usr /bin /etc
28608 files 8.3 GB
54147 files 10.3 GB
93591 files 15.1 GB
127169 files 52.9 GB
175931 files 62.2 GB
213201 files 62.7 GB
Ho wever,itstill takes too lon g to finish.Theresnoreasonwhy all the cal lsto walkDir cantbe
done con cur rently, there byexploit ing paral lelism in the disksystem. Thethirdversionof du,
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
250 CHAPTER 8. GOROUTINES AND CHANNELS
below, cre atesanew goroutine for eachcal l to walkDir.Itusesasync.WaitGroup (§8.5) to
countthe numberofcal lsto walkDir that arestill active, and a clos er goro utine toclos e the
fileSizes ch annel whenthe count erdro pstozero.
gopl.io/ch8/du3
func main() {
// ...determine roots...
// Traverse each root of the file tree in parallel.
fileSizes := make(chan int64)
var n sync.WaitGroup
for _, root := range roots {
n.Add(1)
go walkDir(root, &n, fileSizes)
}
go func() {
n.Wait()
close(fileSizes)
}()
// ...select loop...
}
func walkDir(dir string, n *sync.WaitGroup, fileSizes chan<- int64) {
defer n.Done()
for _, entry := range dirents(dir) {
if entry.IsDir() {
n.Add(1)
subdir := filepath.Join(dir, entry.Name())
go walkDir(subdir, n, fileSizes)
}else {
fileSizes <- entry.Size()
}
}
}
Sincethisprogram createsmanythous and s of goroutinesatits peak, wehavetochange
dirents to use a count ing semaphore topre ventitfro m op eningtoo many files at once, just
as wedid for the web craw ler in Sec tion 8.6:
// sema is a counting semaphore for limiting concurrency in dirents.
var sema = make(chan struct{}, 20)
// dirents returns the entries of directory dir.
func dirents(dir string) []os.FileInfo {
sema <- struct{}{} // acquire token
defer func() { <-sema }() // release token
// ...
Thisversionrunsseveral times fasterthanthe pre vious one,thoug h thereisalot ofvar iabilit y
from systemtosystem.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 8.9. CANCELLATION 251
Exercis e 8.9: Wr ite a versionof du that comp utesand per iodic ally displ ays sep aratetot als for
each ofthe root direc tor ies.
8.9. Cancellation
Sometimesweneed toins truct a goroutine tostopwhatitisdoing , forexample, inaweb
server per for mingacomputation on beh alf ofaclientthathas disconne cte d.
Thereisnoway for onegoroutine toter minateanother direc tly,since thatwou ldleave all its
sh are d var iables in undefine d st ates. Inthe rocketlaunch program (§8.7) wesentasingle
value onachannel named abort,whichthe count dow n goro utine int erprete d as a requestto
stop its elf.But what if we need tocanceltwo goro utines, or an arbit rar y numb er?
Onepossibi lit y mig htbetosendasmanyeventsonthe abort ch annel asthere are goroutines
to cancel. Ifsom e of the goroutineshavealready ter minated thems elves, how ever, our count
wi l l be too large , andour sends will get stuck.Onthe other hand, ifthose goroutineshave
sp awned other goroutines, our countwill betoo small,and som e goro utineswill remain
unawareofthe cancellat ion. Ingeneral,itshardtoknowhow manygoroutinesare wor king
on our beh alf at anygiven mom ent.Moreover, whenagoroutine receivesavalue fro m the
abort ch annel,itcon sumes thatvalue sothatother goroutineswontsee it. For cancellat ion,
what weneed isare liablemechanism to broadcast an event overachannel sothatmany
goro utinescan see it as it occ ursand can later see thatit has occurred.
Re call thatafter a channel has beenclos edand drained ofall sentvalues, subsequentreceive
op erat ions pro ceed immediate ly, yieldingzerovalues. Wecan exploit thistocre ate a bro ad-
cast mechanism: dontsendavalue onthe channel, cl ose it.
We c an add cancellat iontothe du prog ram from the pre vious sec tion wit h afew simple
ch anges. First,wecre ate a cancellat ionchannel onwhichnovalues areeversent, but whose
closureindic ates thatitistimefor the program tostopwhatitisdoing . We als o define a
ut i lit y func tion, cancelled,thatche cks or poll s thecancellat ionstate atthe ins tantitiscal le d.
gopl.io/ch8/du4
var done = make(chan struct{})
func cancelled() bool {
select {
case <-done:
return true
default:
return false
}
}
Next, wecre ate a goroutine thatwill readfro m thestand ard inp ut, whichistypic ally con-
ne cte d to the ter minal.Assoonasany inp utisread(forins tance,the userpress esthe retur n
ke y), thisgoroutine bro adc asts the cancellat ionbyclosingthe done ch annel.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
252 CHAPTER 8. GOROUTINES AND CHANNELS
// Cancel traversal when input is detected.
go func() {
os.Stdin.Read(make([]byte, 1)) // read a single byte
close(done)
}()
No w we need tomakeour goroutinesrespond tothe cancellat ion. Inthe main goroutine,we
add a thirdcas e to the selec t st atement thattries toreceive fro m the done ch annel.The func-
tion retur nsifthiscas e is everselec ted,but beforeitretur nsitmustfirs t drain the fileSizes
ch annel,dis cardingall values until the channel isclos ed. Itdoesthistoens ure thatany active
callsto walkDir canrun tocomplet ionwit houtgetting stuck sendingto fileSizes.
for {
select {
case <-done:
// Drain fileSizes to allow existing goroutines to finish.
for range fileSizes {
// Do nothing.
}
return
case size, ok := <-fileSizes:
// ...
}
}
The walkDir goro utine pol lsthe cancellat ionstatu s when itbeg ins, andretur nswit houtdoing
anyt hing if the statu s is set. Thistur nsall goroutinescre ate d af ter cancellat ionint o no-ops:
func walkDir(dir string, n *sync.WaitGroup, fileSizes chan<- int64) {
defer n.Done()
if cancelled() {
return
}
for _, entry := range dirents(dir) {
// ...
}
}
It mig htbeprotabletopol l thecancellat ionstatu s again wit hin walkDirsloop, toavoid cre-
at inggoroutinesafter the cancellat ionevent.Cancellat ioninv olves a trade-off;aquicker
resp ons e of ten requires moreint rusivechangestoprogram logic. Ensuring thatnoexp ensive
op erat ions everocc ur af ter the cancellat ionevent may requireupdat ingmanyplaces in your
co de,but oftenmostofthe benefitcan beobt ained byche cking for cancellat ioninafew
importantplaces.
Alit tle profilingofthisprogram reveale d that the bot tleneckwas the acquisition of a sema-
ph ore token in dirents.The select belowmakes thisoperat ioncancellable andreduces the
typicalcancellat ionlatency ofthe program fro m hundre dsofmillis econd s to tens:
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 8.10. EXAMPLE: CHAT SERVER 253
func dirents(dir string) []os.FileInfo {
select {
case sema <- struct{}{}: // acquire token
case <-done:
return nil // cancelled
}
defer func() { <-sema }() // release token
// ...read directory...
}
No w,whencancellat ionocc urs, allthe backg round goroutinesquicklystopand the main func-
tion retur ns. Of course,when main returns, a program exits, soitcan behardtotel l amain
func tionthatcle ansupafter its elf fro m on e that does not.Theresahandy trick wecan use
during testing:ifins teadofretur ningfro m main in the event ofcancellat ion, weexe cut e acal l
to panic,thenthe runtime will dumpthe stack ofevery goroutine inthe program. Ifthe main
goro utine isthe onlyone lef t, then ithas cle aned upafter its elf.But ifother goroutines
remain, the y maynot havebeenpro perly cancelled,orperhaps the y have beencancelled but
thecancellat iontakes time; a litt leinv est igat ionmay bewor thw hile. The panic dumpoften
cont ainssufficientinfor mat iontodistinguish these cas es.
Exercis e 8.10: HT TP re quests may becancelled byclosingthe opt ion al Cancel ch annel inthe
http.Request st ruc t. Mo dif y theweb craw ler of Sec tion 8.6 tosup por t cancel lat ion.
Hint:the http.Get conv enience functiondoesnot giveyou anopp ortunity tocustomize a
Request.Ins tead, cre ate the requestusing http.NewRequest,set its Cancel eld, thenper-
form therequestbycal ling http.DefaultClient.Do(req).
Exercis e 8.11: Fo llowing the appro ach of mirroredQuery in Sec tion 8.4.4, implementavar i-
antof fetch that requests several URLs conc urrently. Assoonasthe rs t resp ons e ar r ives,
cancel the other requests.
8.10. Example: Chat Server
Well finish thischapt erwit h achatser ver thatlets several users bro adc asttextualmessagesto
each other.There are fourkinds ofgoroutine inthisprogram. Thereisone ins tance apie ce of
the main and broadcaster goro utines, andfor eachclientconne ction there isone handle-
Conn andone clientWriter goro utine.The bro adc asterisagood illustrat ionofhow select
is used,since ithas torespond tothree dif ferentkinds ofmessages.
Thejob ofthe main goroutine,shown below,istolistenfor and acceptincomingnet wor k con-
ne ction s from clients. For eachone,itcre atesanew handleConn goro utine,justasinthe con-
currentech o server wesaw at the start ofthischapt er.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
254 CHAPTER 8. GOROUTINES AND CHANNELS
gopl.io/ch8/chat
func main() {
listener, err := net.Listen("tcp", "localhost:8000")
if err != nil {
log.Fatal(err)
}
go broadcaster()
for {
conn, err := listener.Accept()
if err != nil {
log.Print(err)
continue
}
go handleConn(conn)
}
}
Next isthe bro adc aster. Its localvar iable clients re cords the cur rentset ofconne cte d clients.
Theonlyinfor mat ionrecorde d ab out eachclientisthe identity ofits out going message chan-
nel,about whichmorelater.
type client chan<- string // an outgoing message channel
var (
entering = make(chan client)
leaving = make(chan client)
messages = make(chan string) // all incoming client messages
)
func broadcaster() {
clients := make(map[client]bool) // all connected clients
for {
select {
case msg := <-messages:
// Broadcast incoming message to all
// clients' outgoing message channels.
for cli := range clients {
cli <- msg
}
case cli := <-entering:
clients[cli] = true
case cli := <-leaving:
delete(clients, cli)
close(cli)
}
}
}
Thebro adc asterlistens onthe global entering and leaving ch annel s forannouncements of
ar r iving and dep arting clients. Whenitreceivesone ofthese events, itupdates the clients
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 8.10. EXAMPLE: CHAT SERVER 255
set, andifthe event was a depar ture, itclos es theclientsout going message channel.The
broadc asterals o listensfor eventsonthe global messages ch annel,towhicheachclientsends
al l itsincomingmessages. Whenthe bro adc asterreceivesone ofthese events, itbro adc asts the
mess age toevery conne cte d client.
No w letslookatthe per-clientgoroutines. The handleConn func tioncre atesanew out going
mess age channel for itsclientand announces the arrival ofthisclienttothe bro adc asterover
the entering ch annel.Thenitreads every lineoftext fro m theclient, sendingeachlinetothe
broadc asteroverthe globalincomingmessage channel,prefixingeachmessage wit h theiden-
tity ofits sender.Oncethere isnot hingmoretoreadfro m theclient, handleConn announces
thedep arture ofthe clientoverthe leaving ch annel and clos es theconne ction.
func handleConn(conn net.Conn) {
ch := make(chan string) // outgoing client messages
go clientWriter(conn, ch)
who := conn.RemoteAddr().String()
ch <- "You are " + who
messages <- who + " has arrived"
entering <- ch
input := bufio.NewScanner(conn)
for input.Scan() {
messages <- who + ": " + input.Text()
}
// NOTE: ignoring potential errors from input.Err()
leaving <- ch
messages <- who + " has left"
conn.Close()
}
func clientWriter(conn net.Conn, ch <-chan string) {
for msg := range ch {
fmt.Fprintln(conn, msg) // NOTE: ignoring network errors
}
}
In addition, handleConn createsaclientWriter goro utine for eachclientthatreceivesmes-
sagesbro adc asttothe clientsout going message channel and writesthemtothe clientsnet-
work conne ction.The clientwritersloopter minates whenthe bro adc asterclos es thechannel
af ter receiving a leaving notication.
Thedispl aybelow shows the ser ver in action wit h twoclients in sep aratewindows onthe
same computer, using netcat to chat:
$gobuild gopl.io/ch8/chat
$gobuild gopl.io/ch8/netcat3
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
256 CHAPTER 8. GOROUTINES AND CHANNELS
$./chat &
$./netcat3
You are 127.0.0.1:64208 $./netcat3
127.0.0.1:64211 has arrived You are 127.0.0.1:64211
Hi!
127.0.0.1:64208: Hi! 127.0.0.1:64208: Hi!
Hi yourself.
127.0.0.1:64211: Hi yourself. 127.0.0.1:64211: Hi yourself.
^C
127.0.0.1:64208 has left
$./netcat3
You are 127.0.0.1:64216 127.0.0.1:64216 has arrived
Welcome.
127.0.0.1:64211: Welcome. 127.0.0.1:64211: Welcome.
^C
127.0.0.1:64211 has left
Whilehosting a chatsessionfor n clients, thisprogram runs2n+2 conc urrentlycommunic at-
inggoroutines, yet itneedsnoexplicitlocking operat ions (§9.2). The clients mapiscon-
ne d to a singlegoroutine,the bro adc aster, soitcannot beaccessedcon cur rently. The only
var iables thatare share d by multiplegoroutinesare channel s andins tances of net.Conn,bot h
of whichare concur rency saf e.Well tal k more about confinement,con cur rency safet y,and the
implic ations ofsharing var iables acrossgoroutines in the next chapt er.
Exercis e 8.12: Make the bro adc asterannouncethe cur rentset ofclients toeachnew arrival.
Thisrequires thatthe clients setand the entering and leaving ch annel s re cordthe client
name too.
Exercis e 8.13: Make the chatser ver disconne ctidleclients, suchasthose thathavesentno
mess agesinthe last ve minut es. Hint: cal ling conn.Close() in anot her goroutine unb locks
ac tive Read callssuch as the one don e by input.Scan().
Exercis e 8.14: Change the chatser versnet wor k prot ocolsothateachclientprovides its name
on ent ering . Us e that nameins teadofthe networ k addresswhenprefixingeachmessage wit h
itssendersidentity.
Exercis e 8.15: Failureofany clientprogram toreaddat a in a timelymanner ult imate lycauses
al l clients toget stuck.Modif y thebro adc astertoskipamessage rat her thanwaitifaclient
wr iterisnot ready toacceptit. Alt ernat ive ly, add buf fer ingtoeachclientsout going message
ch annel sothatmostmessagesare not dro p ped;the bro adc astershoulduse a non-blo cking
send tothischannel.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
9
Concurrency with
Shared Variables
In the pre vious chapt er, wepresent edseveral programsthatuse goroutinesand channel s to
express con cur rency inadirec t andnatural way.How ever, indoing so, weglossedovera
numb erofimp ortantand subtleissues thatprogrammersmustbearinmindwhenwriting
conc urrentcode.
In thischapt er, well takeaclos er lo okatthe mech anics ofcon cur rency.Inpar tic ular, well
pointout som e of the pro blemsass oci ated wit h sh aring var iables amon g mu ltiplegoroutines,
theanalyticaltechniques for recog nizingthose pro blems, andthe patternsfor solv ing them.
Final ly, well explain som e of the technic al dif ferences bet weengoroutinesand operat ingsys-
temthreads.
9.1. RaceConditions
In a sequential program, thatis, a program wit h on lyone goroutine,the steps ofthe program
happ eninthe fami liar exec ution order deter mined bythe program logic. For ins tance,ina
sequence ofstatements, the rs t on e happ ens beforethe secon d on e,and soon. Inaprogram
with two ormoregoroutines, the steps wit hin eachgoroutine happeninthe fami liar order,but
in general wedontknowwhether an event x in one goroutine happens beforeanevent y in
anot her goroutine,orhappens after it, orissimultane ous wit h it.Whenwecannot confidently
saythatone event happ ens before theother,thenthe events x and y are concur rent.
Consider a functionthatwor kscor rec tly inasequential program. That functionis concur-
renc y-s afe if itcontinues towor k correc tly evenwhencal le d conc urrently, thatis, fro m twoor
more goroutineswit h no addition alsynchro nizat ion. Wecan generalize thisnot ion toaset of
257
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
258 CHAPTER 9. CONCURRENCY WITH SHARED VARIABLES
collab orat ingfunctions,suchasthe methodsand operat ions ofapar tic ulartyp e.Atype is
conc urrency-s afe if all its accessiblemet hodsand operat ions are con cur rency-s afe.
We can makeaprogram conc urrency-s afewit houtmak ingevery con crete typ e in that
prog ram conc urrency-s afe. Indeed,con cur rency-s afetyp es arethe exception rat her thanthe
rule,soyou shouldaccessavar iable conc urrentlyonlyifthe documentation for itstyp e says
that thisissafe. Weavoid conc urrentaccesstomostvar iables either by conni n g them toa
singlegoroutine orbymaintainingahig her-le vel invar iantof mu tualexclu sion.Well explain
thes e terms in thischapt er.
In contrast, expor ted package-le vel functions are general lyexp ected tobecon cur rency-s afe.
Sincepackage-le vel var iables cannot beconne d to a singlegoroutine,functions thatmodif y
them mustenforce mut ual exclusion.
Thereare manyreasons a functionmig htnot wor k when cal le d conc urrently, includingdead-
lo ck, live lock, andres ource starvat ion. Wedonthavespace todis cussall ofthem, sowell
fo cus onthe most imp ortantone,the ra c e condit ion.
Arace condit ion isasit uat ioninwhichthe program does not givethe cor rec t resu ltfor som e
interleavingsofthe operat ions ofmultiplegoroutines. Race condit ion s areper nicious because
they may remain latentinaprogram andapp ear infre quently, perhaps onlyunder heavy load
or whenusingcer tain comp ilers, platfor ms, orarc hit ectures. Thismakes themhardto
reproduce anddiagnos e.
It istradition altoexplain the ser iousness ofrace condit ion s thro ugh the metaph oroffinancial
loss, sowell con sider a simpleban k accountprogram.
// Package bank implements a bank with only one account.
package bank
var balance int
func Deposit(amount int) { balance = balance + amount }
func Balance() int { return balance }
(Wecou ldhavewritt enthe bodyofthe Deposit func tionas balance += amount,whichis
equivalent, but the lon g erfor m wi l l simplif y theexplanation.)
Fo r aprogram thistrivial, wecan see ataglancethatany sequence ofcal lsto Deposit and
Balance wi l l give the rig htanswer, thatis, Balance wi l l report the sum ofall amounts
previously dep osite d. Ho wever,ifwecal l thes e func tions not inasequence but con cur rently,
Balance is nolon g erguarante e d to givethe rig htanswer. Con sider the fol low ing two
goro utines, whichrepresent two transactions onajoint ban k account:
// Alice:
go func() {
bank.Deposit(200) // A1
fmt.Println("=", bank.Balance()) // A2
}()
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 9.1. RACE CONDITIONS 259
// Bob:
go bank.Deposit(100) // B
Alice deposits $200, thenche cks her bal ance,whi le Bobdep osits $100. Sincethe steps A1 and
A2 occurcon cur rentlywit h B,wecannot pre dic t theorder in whichthe y happ en. Int uit ive ly, it
mig htseemthatthere are onlythree possibleorder ings, whichwell cal l ‘‘Alice rs t,’’ ‘‘Bob
rs t,’’ and ‘‘Alice/B ob/A lice.’’ Thefol low ing table shows the value ofthe balance var iable after
each step. The quote d st rings represent the print edbal ance slips.
Alice first Bob first Alice/Bob/Alice
000
A1 200 B100 A1 200
A2 "= 200" A1 300 B300
B300 A2 "= 300" A2 "= 300"
In all cas es thefinalbal ance is$300. Theonlyvar iat ioniswhether Alicesbal ance slipincludes
Bobstransactionornot,but the customers are sat isfied eit her way.
Butthisint uit ion iswro ng. There isafourthpossibleout com e,inwhichBobsdep ositocc urs
in the midd le of Alicesdep osit, afterthe bal ance has beenread(balance + amount)but before
it has beenupdated (balance = ...), causingBobstransactiontodis app ear.Thisisbecause
Alicesdep ositoperat ion A1 is reallyasequence oftwo operat ions,areadand a write;cal l them
A1r and A1w.Heresthe pro blemat ic interleaving:
Data race
0
A1r 0 ... = balance + amount
B100
A1w 200 balance = ...
A2 "= 200"
Af ter A1r,the expression balance + amount evaluatesto200, sothisisthe value writt endur-
ing A1w,despit e theint erveningdep osit. The nalbal ance isonly$200. Theban k is $100
rich er at Bobsexp ense.
Thisprogram cont ainsapar tic ularkindofrace condit ion cal le d a data race.Adata race
occurs whene ver two goro utinesaccessthe samevar iable conc urrentlyand atleast one ofthe
accessesisawrite.
Things get evenmessier if the dat a race invo l vesavar iable ofatyp e that islargerthanasingle
machineword, suchasanint erface,astr ing , or a slice.Thiscodeupdates x conc urrentlyto
twoslices ofdif ferentlengt hs:
var x []int
go func() { x = make([]int, 10) }()
go func() { x = make([]int, 1000000) }()
x[999999] = 1 // NOTE: undefined behavior; memory corruption possible!
Thevalue of x in the nalstatement isnot define d;itcou ldbenil,oraslice oflengt h 10, ora
slice oflengt h 1,000,000. But recal l that there are three partstoaslice: the point er, the lengt h,
andthe cap acity.Ifthe point ercom esfro m thefirs t call to make andthe lengt h comesfro m
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
260 CHAPTER 9. CONCURRENCY WITH SHARED VARIABLES
thesecon d, x wouldbea chimera, a slice whose nominal lengt h is 1,000,000 but whose under-
ly ing array has only10elements. Inthisevent uality,storing toelement999,999 wou ldclobb er
an arbit rar y faraway memor y lo cat ion, wit h cons equences thatare imp ossible topre dic t and
hard todebug andlocalize.Thissemanticminefieldiscal le d un dene d behavior andiswel l
know n to C programmers; for tunatelyitisrarely as tro ubles ome in Go as in C.
Ev enthe notion thatacon cur rentprogram isanint erleaving ofseveral sequential programsis
afalse int uit ion.Aswell see inSec tion 9.4, dat a races may haveevenstrangerout com es.
Many programmerseven som e very cle ver oneswi l l occasionallyoffer justication s for
know n data races in their programs: ‘‘thecostofmut ual exclusionistoo hig h,’’ ‘‘this log ic is
on lyfor log ging,’’ ‘‘IdontmindifI dro p some mess ages,’’ andsoon. Theabs ence ofpro blems
on a given comp iler andplatfor m maygivethemfalse confidence.Ago o d rule ofthumb is
that th ere isnosuchthi n g as a benig n data race.Sohow doweavoid dat a races in our
prog rams?
Well rep eat the definition,since itissoimp ortant: A dat a race occ urswhene ver two
goro utinesaccessthe samevar iable conc urrentlyand atleast one ofthe accessesisawrite. It
fo llows fro m this definition thatthere are three ways toavoid a dat a race.
The rs t way isnot towrite the var iable.Con sider the map below,whichislazi lypopu lated as
each key isrequeste d forthe rs t time.If Icon is cal le d sequential ly, the program wor ksfine,
butif Icon is cal le d conc urrently, there isadat a race accessingthe map.
var icons = make(map[string]image.Image)
func loadIcon(name string) image.Image
// NOTE: not concurrency-safe!
func Icon(name string) image.Image {
icon, ok := icons[name]
if !ok {
icon = loadIcon(name)
icons[name] = icon
}
return icon
}
If ins teadweinitialize the map wit h al l ne cessary ent ries beforecre ating addition algoroutines
andnever modif y it again, thenany numberofgoroutinesmay safelycal l Icon conc urrently
since eachonlyreads the map.
var icons = map[string]image.Image{
"spades.png": loadIcon("spades.png"),
"hearts.png": loadIcon("hearts.png"),
"diamonds.png": loadIcon("diamonds.png"),
"clubs.png": loadIcon("clubs.png"),
}
// Concurrency-safe.
func Icon(name string) image.Image { return icons[name] }
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 9.1. RACE CONDITIONS 261
In the exampleabove , the icons var iable isassig ned dur ingpackageinitializat ion, which
happ ens before theprograms main func tionstartsrunning. Onceinitialize d, icons is never
mo dified.Dat a st ruc tures thatare never modified orare immut ableare inherentlycon cur-
renc y-s afeand need nosynchro nizat ion. But obv iou sly wecantuse thisappro ach ifupdates
areess ent ial, as wit h aban k account.
Thesecon d way toavoid a dat a race istoavoid accessingthe var iable fro m mu ltiple
goro utines. Thisisthe appro ach taken bymanyofthe programsinthe pre vious chapt er. For
example, the main goroutine inthe con cur rentweb craw ler (§8.6) isthe solegoroutine that
accessesthe seen map, and the broadcaster goro utine inthe chatser ver (§8.10) isthe only
goro utine thataccessesthe clients map. These var iables are conne d to a singlegoroutine.
Sinceother goroutinescannot accessthe var iable direc tly,the y mu stuse a channel tosendthe
confininggoroutine a requesttoquer y or update the var iable.Thisiswhatismeant bythe Go
mant ra ‘‘Do not communic atebysharing memor y;ins tead, share memor y by communic at-
ing.’’ Agoroutine thatbro kersaccesstoaconne d var iable usingchannel requests iscal le d a
moni tor goroutine forthatvar iable.For example, the broadcaster goro utine monitors access
to the clients map.
Heresthe ban k examplerewritt enwit h the balance var iable confine d to a monit or goro utine
called teller:
gopl.io/ch9/bank1
// Package bank provides a concurrency-safe bank with one account.
package bank
var deposits = make(chan int) // send amount to deposit
var balances = make(chan int) // receive balance
func Deposit(amount int) { deposits <- amount }
func Balance() int {return <-balances }
func teller() {
var balance int // balance is confined to teller goroutine
for {
select {
case amount := <-deposits:
balance += amount
case balances <- balance:
}
}
}
func init() {
go teller() // start the monitor goroutine
}
Ev enwhenavar iable cannot beconne d to a singlegoroutine for itsent ire lifet ime, conne-
ment may still beasolut ion tothe pro blem ofcon cur rentaccess. For example, itscommonto
sh are a var iable bet weengoroutinesinapip elinebypassingits addressfro m on e st age tothe
next overachannel.Ifeachstage ofthe pip elineref rainsfro m accessingthe var iable after
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
262 CHAPTER 9. CONCURRENCY WITH SHARED VARIABLES
sendingittothe next stage , then allaccessestothe var iable aresequential.Ineffec t,the var i-
able isconne d to one stage ofthe pip eline, thenconne d to the next, andsoon. Thisdis ci-
plineissom etimescal le d serial connement.
In the examplebelow, Cakesare ser ial lyconne d,firs t to the baker goro utine,thentothe
icer goro utine:
type Cake struct{ state string }
func baker(cooked chan<- *Cake) {
for {
cake := new(Cake)
cake.state = "cooked"
cooked <- cake // baker never touches this cake again
}
}
func icer(iced chan<- *Cake, cooked <-chan *Cake) {
for cake := range cooked {
cake.state = "iced"
iced <- cake // icer never touches this cake again
}
}
Thethirdway toavoid a dat a race istoallow manygoroutinestoaccessthe var iable,but only
on e at a time. Thisappro ach isknown as mu tualexclu sion andisthe subjec t of the next
section.
Exercis e 9.1: Addafunction Withdraw(amount int) bool to the gopl.io/ch9/bank1
prog ram. Theresultshouldindic atewhether the transactionsucce e de d or fai le d duetoins uf-
ficientfunds.The mess age senttothe monitorgoroutine mustcontain bot h theamountto
withdraw andanew channel overwhichthe monitorgoroutine can sendthe boole an resu lt
back to Withdraw.
9.2. Mutual Exclusion: sync.Mutex
In Sec tion 8.6, weusedabuf fered channel asacounting semap hore to ens ure thatnomore
than 20 goroutinesmade simultane ous HTTPrequests. Wit h thesameide a, we can use a
ch annel ofcap acity 1 toens ure thatatmostone goroutine accessesashare d var iable at a time.
Asemaphore thatcountsonlyto1iscal le d a binary semap hore.
gopl.io/ch9/bank2
var (
sema =make(chan struct{}, 1) // a binary semaphore guarding balance
balance int
)
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 9.2. MUTUAL EXCLUSION: SYNC.MUTEX 263
func Deposit(amount int) {
sema <- struct{}{} // acquire token
balance = balance + amount
<-sema // release token
}
func Balance() int {
sema <- struct{}{} // acquire token
b:=balance
<-sema // release token
return b
}
Thispattern of mu tualexclu sion is sousefulthatitissup por ted direc tly bythe Mutex type
from the sync package. Its Lock method acquires the token (cal le d a lock)and its Unlock
method releasesit:
gopl.io/ch9/bank3
import "sync"
var (
mu sync.Mutex // guards balance
balance int
)
func Deposit(amount int) {
mu.Lock()
balance = balance + amount
mu.Unlock()
}
func Balance() int {
mu.Lock()
b:=balance
mu.Unlock()
return b
}
Each timeagoroutine accessesthe var iables ofthe ban k (just balance here), itmustcal l the
mu texs Lock method toacquireanexc lusivelock. Ifsom e ot her goroutine has acquired the
lo ck, thisoperat ionwill blo ckunt i l theother goroutine cal ls Unlock andthe lockbecom es
avai lable again. Themut ex guards theshare d var iables. Byconvent ion,the var iables guarde d
by a mut exare declare d immediate lyafter the declarat ionofthe mut exits elf.Ifyou deviate
from this, besuretodocumentit.
Thereg ionofcodebet ween Lock and Unlock in whichagoroutine isfre e to readand modif y
theshare d var iables iscal le d a cr iti cal sec tion.The lockholderscal l to Unlock happ ens before
anyother goroutine can acquirethe lockfor itself.Itisess ent ial thatthe goroutine release the
lo ckonceitis finishe d,onall pathsthrough the function, includingerror pat hs.
Theban k prog ram ab ove exemp lifies a common con cur rency pattern.Asetofexp orted func-
tion s encapsulates one ormorevar iables sothatthe onlyway toaccessthe var iables isthrough
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
264 CHAPTER 9. CONCURRENCY WITH SHARED VARIABLES
thes e func tions (or met hods, for the var iables ofanobj e ct). Eachfunctionacquires a mut ex
lo ckatthe beg inningand releasesitatthe end,there byens uring thatthe shared var iables are
notaccessedcon cur rently. Thisarrangement offunctions,mut exlock, andvar iables iscal le d
a moni tor.(Thisolder use ofthe word ‘‘monitor’’ inspired the ter m ‘‘monitorgoroutine.’’ Both
us esshare the meaningofabro ker thatens uresvar iables areaccessedsequential ly.)
Sincethe criticalsec tion s in the Deposit and Balance func tions are soshortasingleline, no
branchingcalling Unlock at the end isstraig htfor ward. Inmorecomplex crit icalsec tion s,
especi ally those inwhicherror s mu stbedealt wit h by retur ningearly,itcan behardtotel l that
callsto Lock and Unlock arestr ictly paired onall paths. Gos defer st atement com estothe
rescue: bydefer r ingacal l to Unlock,the criticalsec tion imp licitlyextends tothe end ofthe
currentfunction, fre eingusfro m having toremembertoins ert Unlock callsinone ormore
pl aces far fro m thecal l to Lock.
func Balance() int {
mu.Lock()
defer mu.Unlock()
return balance
}
In the exampleabove , the Unlock exec utes af ter theretur n st atement has readthe value of
balance,sothe Balance func tioniscon cur rency-s afe. Asa bonus,wenolon g erneed the
lo cal variable b.
Furthermore, a defer red Unlock wi l l runevenifthe criticalsec tion panics, whichmay be
important in programsthatmakeuse of recover (§5.10). A defer is marg inallymoreexp en-
sive thananexplicitcal l to Unlock,but not enoug h to justify lesscle ar co de.Asalways wit h
conc urrentprograms, favor clarity and resist premature opt imizat ion. Where possible, use
defer andlet crit icalsec tion s extendtothe end ofafunction.
Consider the Withdraw func tionbelow.Onsuccess, itreduces the bal ance bythe specified
amount and retur ns true.But ifthe accountholds ins ufficientfunds for the transaction,
Withdraw restores the bal ance andretur ns false.
// NOTE: not atomic!
func Withdraw(amount int) bool {
Deposit(-amount)
if Balance() < 0 {
Deposit(amount)
return false // insufficient funds
}
return true
}
Thisfunctionevent ual lygives the cor rec t resu lt, but it has a nasty side effe ct. Whenanexces-
sive wit hdrawal isatt emp ted,the bal ance transientlydipsbelow zero. Thismay cause a con-
currentwit hdrawal for a modestsum tobespuriou sly rej e cte d. So ifBob tries tobuy a sports
car, Alice cantpay for her mor ningcof fee.The pro blem isthat Withdraw is not at o mic:it
consists ofasequence ofthree sep arateoperat ions,eachofwhichacquires andthenreleases
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 9.2. MUTUAL EXCLUSION: SYNC.MUTEX 265
themut exlock, but not hinglocks the whole sequence.
Ideally, Withdraw shouldacquirethe mut exlockoncearo und the whole operat ion. How ever,
this att emp t wontwor k:
// NOTE: incorrect!
func Withdraw(amount int) bool {
mu.Lock()
defer mu.Unlock()
Deposit(-amount)
if Balance() < 0 {
Deposit(amount)
return false // insufficient funds
}
return true
}
Deposit tr ies toacquirethe mut exlockasecon d time bycal ling mu.Lock(),but because
mu tex locksare not re-entrantitsnot possibletolockamut exthatsalready locke dthis
le adstoadeadlockwhere not hingcan pro ceed,and Withdraw blocks forever.
Thereisagood reasonGosmut exesare not re-entrant. Thepur pos e of a mut existoens ure
that certain invar iants ofthe shared var iables aremaintained atcriticalpointsdur ingprogram
exec ution.One ofthe invar iants is ‘‘no goroutine isaccessingthe shared var iables,’’ butthere
maybeaddition alinvar iants specifictothe dat a st ruc tures thatthe mut exguard s.Whena
goro utine acquires a mut exlock, itmay assume thatthe invar iants hold. Whi le it holds the
lo ck, itmay updatethe shared var iables sothatthe invar iants aretemporar ily violate d.
Ho wever,whenitreleasesthe lock, itmustguarante e that order has beenrestore d andthe
invar iants holdonceagain. Alt hough a re-entrantmut exwou ldens ure thatnoother
goro utinesare accessingthe shared var iables, itcannot pro tec t theaddition alinvar iants of
thos e var iables.
Acommonsolut ion istodiv ide a functionsuchas Deposit into two:anunexp orted function,
deposit,thatassumesthe lockisalready heldand doesthe realwor k,and anexp orted func-
tion Deposit that acquires the lockbeforecal ling deposit.Wecan thenexpress Withdraw in
termsof deposit li kethis:
func Withdraw(amount int) bool {
mu.Lock()
defer mu.Unlock()
deposit(-amount)
if balance < 0 {
deposit(amount)
return false // insufficient funds
}
return true
}
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
266 CHAPTER 9. CONCURRENCY WITH SHARED VARIABLES
func Deposit(amount int) {
mu.Lock()
defer mu.Unlock()
deposit(amount)
}
func Balance() int {
mu.Lock()
defer mu.Unlock()
return balance
}
// This function requires that the lock be held.
func deposit(amount int) { balance += amount }
Of course,the deposit func tionshown hereissotrivialthatarealist ic Withdraw func tion
wouldntbot her cal lingit, but non ethelessitillustrates the princip le.
Encapsulat ion(§6.6), byreducingunexp ected int erac tions ina program, helps usmaintain
data str uctureinvar iants. For the samereason, encapsulat ionals o helps usmaintain conc ur-
renc y invar iants. Whenyou use a mut ex, makesurethatbot h it and the var iables itguard s are
notexp orted,whether the y arepackage-le vel var iables orthe elds ofastr uct.
9.3. Read/WriteMutexes: sync.RWMutex
In a tofanxiet y af ter seeinghis $100 depositvanish wit houtatrace,Bob writesaprogram to
ch eck his ban k balancehundredsoftimes a secon d.Herunsitathom e,atwor k,and onhis
ph one.The ban k notices thatthe incre asedtrafficisdel aying dep osits andwit hdrawals,
becaus e al l the Balance re quests run sequential ly, holdingthe lockexc lusivelyand temporar-
ilypre venting other goroutinesfro m running.
Sincethe Balance func tiononlyneedsto re ad thestate ofthe var iable,itwou ldinfac t be safe
formultiple Balance callstorun conc urrently, solon g as no Deposit or Withdraw call isrun-
ning. Inthisscenar io we need a speci al kind oflockthatallowsread-onlyoperat ions to
proceed inparal lelwit h each other,but write operat ions tohavefullyexc lusiveaccess. This
lo ckiscal le d a mu ltipl e re aders, single writer lo ck, and in Go itsprovide d by sync.RWMutex:
var mu sync.RWMutex
var balance int
func Balance() int {
mu.RLock() // readers lock
defer mu.RUnlock()
return balance
}
The Balance func tionnow cal lsthe RLock and RUnlock methodstoacquireand release a
re aders or sh are d lo ck. The Deposit func tion, whichisunchange d,cal lsthe mu.Lock and
mu.Unlock methodstoacquireand release a writ er or ex clu siv e lo ck.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 9.4. MEMORYSYNCHRONIZATION 267
Af ter thischange , most ofBobs Balance re quests run in paral lelwit h each other and finish
more quickly. The lockisavai lable for moreofthe time, and Deposit re quests can pro ceed in
atimelymanner.
RLock canbeusedonlyifthere are nowritestoshare d var iables in the criticalsec tion.Ingen-
eral,weshouldnot assume that logically re ad-onlyfunctions ormet hodsdontals o up date
some var iables. For example, a met hod thatapp earstobeasimpleaccessormig htals o incre-
ment anint ernal usage count er, orupdateacache sothatrep eat cal lsare faster. Ifindou bt,
us e an exclusive Lock.
Itsonlyprotabletouse an RWMutex when mostofthe goroutinesthatacquirethe lockare
re aders, andthe lockisunder cont ent ion,thatis, goroutinesroutinelyhavetowaittoacquire
it.An RWMutex re quires morecomplex internal bookkeeping , making itslowerthanaregu lar
mu tex for uncontended locks.
9.4. Memory Synchronization
Yo u maywon der why the Balance method needsmut ual exclusion, either channel-b ased or
mu tex-b ased.After all, unlike Deposit,itcon sists onlyofasingleoperat ion, sothere isno
dangerofanother goroutine exe cut ing ‘‘in the midd le’’ of it.There are two reasons we need a
mu tex. The rs t is thatitsequ allyimp ortantthat Balance notexe cut e in the midd le of som e
ot her operat ionlike Withdraw.The secon d (andmoresubtle) reasonisthatsynchro nizat ion
is about morethanjustthe order ofexe cut ion of multiplegoroutines; synchronizat ionals o
af fec tsmemor y.
In a moder n comp uterthere may bedozensofpro cessors,eachwit h itsown local cache ofthe
main memor y.For efficiency,writestomemor y arebuf fered wit hin eachpro cessorand
ushe d outtomain memor y on lywhennecessary.The y mayevenbecommitt edtomain
memory inadif ferentorder thanthe y were writt enbythe writing goro utine.Synchro nizat ion
pr imitiveslikechannel communic ations and mut exoperat ions cause the pro cessortoflush
outand commitall its accumulated writessothatthe effec tsofgoroutine exe cut ion up tothat
pointare guarante e d to bevisible togoroutinesrunningonother pro cessors.
Consider the possibleout putsofthe fol low ing snipp etofcode:
var x, y int
go func() {
x=1 // A1
fmt.Print("y:", y, " ") // A2
}()
go func() {
y=1 // B1
fmt.Print("x:", x, " ") // B2
}()
Sincethese two goro utinesare con cur rentand accessshare d var iables wit houtmut ual
exclusion, there isadat a race,soweshouldnot besur prisedthatthe program isnot
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
268 CHAPTER 9. CONCURRENCY WITH SHARED VARIABLES
deter minist ic. Wemig htexp ect ittoprint any one ofthese fourresults, whichcor respond to
intuit ive int erleavingsofthe lab ele d st atementsofthe program:
y:0 x:1
x:0 y:1
x:1 y:1
y:1 x:1
Thefourthlinecou ldbeexplained bythe sequence A1,B1,A2,B2 or by B1,A1,A2,B2,for
example. How ever, these two out com esmig htcom e as a sur prise:
x:0 y:0
y:0 x:0
butdep endingonthe compi ler,CPU,and manyother factors,the y canhappentoo.What
possible interleaving ofthe fourstatementscou ldexplain them?
Wi thin a singlegoroutine,the effec tsofeachstatement are guarante e d to occ ur in the order of
exec ution;goroutinesare sequ ent ial lyconsi stent.But inthe abs ence ofexplicitsynchro niza-
tion usingachannel ormut ex, there isnoguarante e that eventsare seeninthe sameorder by
al l goro utines. Alt hough goroutine A mu stobs erve the effec t of the write x=1before itreads
thevalue of y,itdoesnot necessarily obs erve the write to y done bygoroutine B,so A may
pr int a stale value of y.
It istempt ing totry tounderst and con cur rency asifitcor respond s to some interleaving ofthe
st atementsofeachgoroutine,but asthe exampleabove shows, thisisnot how a moder n com-
pi ler orCPU wor ks. Because the assig nmentand the Print refertodif ferentvar iables, a com-
pi ler may conclude thatthe order ofthe two statementscannot affec t theresult, andswap
them. Ifthe two goro utinesexe cut e on dif ferentCPUs, eachwit h itsown cach e, wr itesbyone
goro utine are not visible tothe other goroutines Print until the cachesare synchro nize d with
main memor y.
Al l thes e conc urrency pro blemscan beavoide d by the con sistent use ofsimple, est ablishe d
patt erns. Where possible, conne var iables toasinglegoroutine;for all other variables, use
mu tualexc lusion.
9.5. Lazy Initialization: sync.Once
It isgood prac tice todefer an expensive initializat ionstepunt i l themom ent itisneeded.Ini-
tializingavar iable upfro ntincre asesthe start-uplatency ofaprogram andisunnecessary if
exec ution doesntalways reach the par t of the program thatusesthatvar iable.Letsretur n to
the icons var iable wesaw earlier in the chapt er:
var icons map[string]image.Image
Thisversionof Icon us es lazy ini tializati on:
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 9.5. LAZY INITIALIZATION: SYNC.ONCE 269
func loadIcons() {
icons = map[string]image.Image{
"spades.png": loadIcon("spades.png"),
"hearts.png": loadIcon("hearts.png"),
"diamonds.png": loadIcon("diamonds.png"),
"clubs.png": loadIcon("clubs.png"),
}
}
// NOTE: not concurrency-safe!
func Icon(name string) image.Image {
if icons == nil {
loadIcons() // one-time initialization
}
return icons[name]
}
Fo r avar iable accessedbyonlyasinglegoroutine,wecan use the pattern above , butthispat-
tern isnot safeif Icon is cal le d conc urrently. Likethe ban ksoriginal Deposit func tion, Icon
consists ofmultiplesteps: ittests whether icons is nil,thenitloads the icons,thenitupdates
icons to a non-ni l value.Int uit ion mig htsug gestthatthe wor stpossibleout com e of the race
condit ion above isthatthe loadIcons func tioniscal le d several times. Whi le thefirs t
goro utine isbusyloadingthe icons,another goroutine ent ering Icon wouldfind the var iable
st i l l equalto nil,and wou ldals o call loadIcons.
Butthisint uit ion isals o wron g.(We hop e that bynow you are developinganew int uit ion
ab out con cur rency,thatint uit ion s ab out con cur rency are not tobetrusted!) Recal l thedis-
cussionofmemor y from Sec tion 9.4. Inthe abs ence ofexplicitsynchro nizat ion, the compi ler
andCPU arefre e to reorder accessestomemor y in anynumberofways, solon g as the beh av-
iorofeachgoroutine issequential lycon sistent.One possiblereorder ingofthe statementsof
loadIcons is shown below.Itstoresthe emp tymap in the icons var iable beforepopu lat ingit:
func loadIcons() {
icons = make(map[string]image.Image)
icons["spades.png"] = loadIcon("spades.png")
icons["hearts.png"] = loadIcon("hearts.png")
icons["diamonds.png"] = loadIcon("diamonds.png")
icons["clubs.png"] = loadIcon("clubs.png")
}
Cons equently, a goroutine nding icons to benon-ni l maynot assume thatthe initializat ion
of the var iable iscomplete.
Thesimplestcor rec t way toens ure thatall goroutinesobs erve the effec tsof loadIcons is to
sy nchro nize themusingamut ex:
var mu sync.Mutex // guards icons
var icons map[string]image.Image
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
270 CHAPTER 9. CONCURRENCY WITH SHARED VARIABLES
// Concurrency-safe.
func Icon(name string) image.Image {
mu.Lock()
defer mu.Unlock()
if icons == nil {
loadIcons()
}
return icons[name]
}
Ho wever,the costofenforcingmut ual lyexc lusiveaccessto icons is thattwo goro utinescan-
notaccessthe var iable conc urrently, evenoncethe var iable has beensafelyinitialize d andwill
ne ver bemodified again. Thissug gests a multiple-readerslock:
var mu sync.RWMutex // guards icons
var icons map[string]image.Image
// Concurrency-safe.
func Icon(name string) image.Image {
mu.RLock()
if icons != nil {
icon := icons[name]
mu.RUnlock()
return icon
}
mu.RUnlock()
// acquire an exclusive lock
mu.Lock()
if icons == nil { // NOTE: must recheck for nil
loadIcons()
}
icon := icons[name]
mu.Unlock()
return icon
}
Thereare now two criticalsec tion s.The goroutine rs t acquires a reader lock,con sults the
map, thenreleasesthe lock. Ifanent rywas found (thecommoncas e), it isretur ned.Ifno
entr y was found,the goroutine acquires a writerlock. Thereisnoway toupg rade ashare d
lo cktoanexc lusiveone wit houtfirs t re leasingthe shared lock, sowemustreche ckthe icons
var iable in cas e anot her goroutine already initialize d it inthe int erim.
Thepattern above gives usgre atercon cur rency but iscomplex andthu s er ror-pro ne.
Fo rtunate ly, the sync packageprovides a speci alizedsolut ion tothe pro blem ofone-t imeini-
tializat ion: sync.Once.Con ceptu ally, a Once consists ofamut exand a boole an var iable that
re cords whether initializat ionhas taken place; the mut exguard s both theboole an andthe
clientsdat a st ruc tures. Thesolemet hod, Do,accepts the initializat ionfunctionasits argu-
ment.Letsuse Once to simplif y the Icon func tion:
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 9.6. THE RACE DETECTOR 271
var loadIconsOnce sync.Once
var icons map[string]image.Image
// Concurrency-safe.
func Icon(name string) image.Image {
loadIconsOnce.Do(loadIcons)
return icons[name]
}
Each cal l to Do(loadIcons) lo cks the mut exand che cks the boole an var iable.Inthe rs t call,
in whichthe var iable isfalse, Do calls loadIcons andsets the var iable totrue. Subsequent
callsdonot hing, but the mut exsynchro nizat ionens uresthatthe effec tsof loadIcons on
memory (sp ecifically, icons)becom e visibletoall goroutines. Using sync.Once in thisway,
we can avoidsharing var iables wit h ot her goroutinesunt i l they havebeenpro perly con-
st ruc ted.
Exercis e 9.2: Re writ e the PopCount examplefro m Section2.6.2 sothatitinitializes the looku p
tableusing sync.Once thefirs t time itisneeded.(Re alist ically, the costofsynchro nizat ion
wouldbepro hibit ive for a small and hig hly opt imize d func tionlike PopCount.)
9.6. The RaceDet ector
Ev enwit h thegre atest ofcare, itsall too easy tomakecon cur rency mistakes. For tunately, the
Go runtime and toolchain areequip ped wit h asop histicate d andeasy-to-use dynamic analysis
to ol, the ra c e dete ctor.
Ju stadd the -race flag toyour go build, go run,or go test command. Thiscausesthe com-
pi ler tobui ld amodified versionofyourapp lic ationortestwit h addition alins trument ation
that effe ctive lyrecords all accessestoshare d var iables thatocc urre d during exe cut ion,along
with theidentity ofthe goroutine thatreadorwro tethe var iable.Inaddition,the modified
prog ram re cords all synchro nizat ionevents, suchas go st atements, channel operat ions,and
callsto (*sync.Mutex).Lock, (*sync.WaitGroup).Wait,and soon. (Thecompleteset of
sy nchro nizat ioneventsisspecified bythe TheGoMemor y Mo d el do cumentthataccomp anies
thelangu agespecification.)
Therace detec tor studies thisstreamofevents, look ing for cas es in whichone goroutine reads
or writesashare d var iable thatwas mostrecentlywritt enbyadif ferentgoroutine wit houtan
interveningsynchro nizat ionoperat ion. Thisindic ates a conc urrentaccesstothe shared var i-
able,and thu s adat a race.The toolprintsarep ort thatincludes the identity ofthe var iable,
andthe stacksofactivefunctioncal lsinthe readinggoroutine and the writing goro utine.This
is usuallysufficienttopinpointthe pro blem. Sec tion 9.7 cont ainsanexampleofthe race
detec tor inaction.
Therace detec tor rep ortsall dat a races thatwereactuallyexe cut ed. How ever, itcan only
detec t race condit ion s that occ ur during a run; itcannot prove thatnon e wi l l ever occ ur.For
best results, makesurethatyourtests exercise yourpackages usingcon cur rency.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
272 CHAPTER 9. CONCURRENCY WITH SHARED VARIABLES
Duetoext ra bookkeeping , aprogram bui ltwit h race detec tion needsmoretimeand memory
to run, but the overhe adistolerable evenfor manypro duc tionjobs. For inf requentlyocc ur-
ring race condit ion s,letting the race detec tor doits job can savehours ordays ofdebug ging.
9.7. Example: ConcurrentNon-Blocking Cache
In thissec tion,well bui ld a concur rentnon-blocki n g cache, an abstrac tionthatsolves a
problem thatarisesoften in real-wor ldcon cur rentprogramsbut isnot wel l addressedby
exist ing librar ies. Thisisthe pro blem of memoizing afunction, thatis, cachingthe resultofa
func tionsothatitneed becompute d on lyonce. Our solut ion will becon cur rency-s afeand
wi l l avoidthe content ion ass oci ated wit h desig nsbas edonasinglelockfor the whole cache.
Well use the httpGetBody func tionbelow asanexampleofthe typ e of functionwemig ht
wanttomemoize.Itmakes an HTTP GETrequestand reads the requestbody. Cal lstothis
func tionare relative lyexp ensive , so wedliketoavoid rep eat ingthemunnecessarily.
func httpGetBody(url string) (interface{}, error) {
resp, err := http.Get(url)
if err != nil {
return nil, err
}
defer resp.Body.Close()
return ioutil.ReadAll(resp.Body)
}
The nallinehides a minor subtlet y. ReadAll returnstwo results, a []byte andan error,but
since these are assig nable tothe declare d resu lttyp es of httpGetBodyinterface{} and
error,respectivelywe can retur n theresultofthe cal l withoutfur therado.Wechose this
return typ e for httpGetBody so thatitconformstothe typ e of functions thatour cache is
desig ned tomemoize.
Heresthe rs t draf t of the cache:
gopl.io/ch9/memo1
// Package memo provides a concurrency-unsafe
// memoization of a function of type Func.
package memo
// A Memo caches the results of calling a Func.
type Memo struct {
fFunc
cache map[string]result
}
// Func is the type of the function to memoize.
type Func func(key string) (interface{}, error)
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 9.7. EXAMPLE: CONCURRENT NON-BLOCKING CACHE 273
type result struct {
value interface{}
err error
}
func New(f Func) *Memo {
return &Memo{f: f, cache: make(map[string]result)}
}
// NOTE: not concurrency-safe!
func (memo *Memo) Get(key string) (interface{}, error) {
res, ok := memo.cache[key]
if !ok {
res.value, res.err = memo.f(key)
memo.cache[key] = res
}
return res.value, res.err
}
A Memo inst anceholds the function f to memoize,oftyp e Func,and the cache,whichisa
mappingfro m st rings to results. Each result is simply the pair ofresults retur ned byacal l
to favalue and an error.Well showseveral variation s of Memo as the desig n prog resses, but
al l wi l l sh are these basic aspects.
An exampleofhow touse Memo appearsbelow.For eachelementinastreamofincoming
URLs, wecal l Get,log gingthe latency ofthe cal l andthe amountofdat a it retur ns:
m:=memo.New(httpGetBody)
for url := range incomingURLs() {
start := time.Now()
value, err := m.Get(url)
if err != nil {
log.Print(err)
}
fmt.Printf("%s, %s, %d bytes\n",
url, time.Since(start), len(value.([]byte)))
}
We can use the testing package(thetopic ofChapt er11) tosystematicallyinv est igatethe
ef fec t of memoizat ion. Fro m thetestout put below,wesee thatthe URL streamcontains
dup lic ates, andthatalt hough the rs t call to (*Memo).Get foreachURL takes hundredsof
mi l lisecond s,the secon d re questretur nsthe sameamountofdat a in under a millis econd.
$gotest -v gopl.io/ch9/memo1
=== RUN Test
https://golang.org, 175.026418ms, 7537 bytes
https://godoc.org, 172.686825ms, 6878 bytes
https://play.golang.org, 115.762377ms, 5767 bytes
http://gopl.io, 749.887242ms, 2856 bytes
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
274 CHAPTER 9. CONCURRENCY WITH SHARED VARIABLES
https://golang.org, 721ns, 7537 bytes
https://godoc.org, 152ns, 6878 bytes
https://play.golang.org, 205ns, 5767 bytes
http://gopl.io, 326ns, 2856 bytes
--- PASS: Test (1.21s)
PASS
ok gopl.io/ch9/memo1 1.257s
Thistestexe cut esall cal lsto Get sequential ly.
SinceHTTPrequests areagre atopp ortunity for paral lelism, letschange the testsothatit
makesall requests conc urrently. The testusesasync.WaitGroup to waitunt i l thelastrequest
is completebeforeretur ning.
m:=memo.New(httpGetBody)
var n sync.WaitGroup
for url := range incomingURLs() {
n.Add(1)
go func(url string) {
start := time.Now()
value, err := m.Get(url)
if err != nil {
log.Print(err)
}
fmt.Printf("%s, %s, %d bytes\n",
url, time.Since(start), len(value.([]byte)))
n.Done()
}(url)
}
n.Wait()
Thetestrunsmuchfaster, but unfor tunatelyitisunlikelytowor k correc tly all the time. We
maynot ice unexp ected cache misses, orcache hitsthatretur n incorrec t values, oreven
crashes.
Wo rse,itislikelytowor k correc tly some of the time, sowemay not evennot ice thatithas a
problem. But ifwerun itwit h the -race flag , therace detec tor (§9.6) often printsarep ort
such asthisone:
$gotest -run=TestConcurrent -race -v gopl.io/ch9/memo1
=== RUN TestConcurrent
...
WARNING: DATA RACE
Write by goroutine 36:
runtime.mapassign1()
~/go/src/runtime/hashmap.go:411 +0x0
gopl.io/ch9/memo1.(*Memo).Get()
~/gobook2/src/gopl.io/ch9/memo1/memo.go:32 +0x205
...
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 9.7. EXAMPLE: CONCURRENT NON-BLOCKING CACHE 275
Previous write by goroutine 35:
runtime.mapassign1()
~/go/src/runtime/hashmap.go:411 +0x0
gopl.io/ch9/memo1.(*Memo).Get()
~/gobook2/src/gopl.io/ch9/memo1/memo.go:32 +0x205
...
Found 1 data race(s)
FAIL gopl.io/ch9/memo1 2.393s
Thereference to memo.go:32 te lls us thattwo goro utineshaveupdated the cache map
withoutany int erveningsynchro nizat ion. Get is not con cur rency-s afe: ithas a dat a race.
28 func (memo *Memo) Get(key string) (interface{}, error) {
29 res, ok := memo.cache[key]
30 if !ok {
31 res.value, res.err = memo.f(key)
32 memo.cache[key] =res
33 }
34 return res.value, res.err
35 }
Thesimplestway tomakethe cache con cur rency-s afeistouse monit or-b ased synchro niza-
tion.All weneed to do is add a mut extothe Memo,acquirethe mut exlock at the start of Get,
andrelease itbefore Get returns, sothatthe two cache op erat ions occ ur within the critical
section:
gopl.io/ch9/memo2
type Memo struct {
fFunc
mu sync.Mutex // guards cache
cache map[string]result
}
// Get is concurrency-safe.
func (memo *Memo) Get(key string) (value interface{}, err error) {
memo.mu.Lock()
res, ok := memo.cache[key]
if !ok {
res.value, res.err = memo.f(key)
memo.cache[key] = res
}
memo.mu.Unlock()
return res.value, res.err
}
No w therace detec tor issilent, evenwhenrunningthe tests conc urrently. Unfor tunatelythis
ch ange to Memo re versesour earlier per for mance gains. Byholdingthe lockfor the durat ionof
each cal l to f, Get serializes allthe I/O operat ions we int ended toparal lelize.Whatweneed is
a non-blocki n g cach e, on e that does not ser ialize cal lstothe functionitmemoizes.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
276 CHAPTER 9. CONCURRENCY WITH SHARED VARIABLES
In the next implementation of Get,below,the cal linggoroutine acquires the locktwice: once
forthe looku p,and thenasecon d time for the updateifthe looku p returned not hing. In
between, other goroutinesare fre e to use the cache.
gopl.io/ch9/memo3
func (memo *Memo) Get(key string) (value interface{}, err error) {
memo.mu.Lock()
res, ok := memo.cache[key]
memo.mu.Unlock()
if !ok {
res.value, res.err = memo.f(key)
// Between the two critical sections, several goroutines
// may race to compute f(key) and update the map.
memo.mu.Lock()
memo.cache[key] = res
memo.mu.Unlock()
}
return res.value, res.err
}
Theper for mance improv esagain, but now wenot ice thatsom e URLs arebeingfetch edtwice.
Thishappens whentwo ormoregoroutinescal l Get forthe sameURL at aboutthe sametime.
Both con sultthe cache,find no value there , andthencal l theslowfunction f.Thenbot h of
them updatethe map wit h theresultthe y obtained.One ofthe results isoverwritt enbythe
ot her.
Ideallywedliketoavoid thisredundant wor k.Thisfeature issom etimescal le d du p licate
sup pressi on.Inthe versionof Memo below, eachmap elementisapoint ertoan entry st ruc t.
Each entry cont ainsthe memoize d resu ltofacal l to the function f,asbefore, but it addition-
al lycontainsachannel cal le d ready.Justafter the entrys result hasbeenset, thischannel
wi l l be clos ed, to broadcast (§8.9) toany other goroutinesthatitisnow safefor themtoread
theresultfro m the entry.
gopl.io/ch9/memo4
type entry struct {
res result
ready chan struct{} // closed when res is ready
}
func New(f Func) *Memo {
return &Memo{f: f, cache: make(map[string]*entry)}
}
type Memo struct {
fFunc
mu sync.Mutex // guards cache
cache map[string]*entry
}
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 9.7. EXAMPLE: CONCURRENT NON-BLOCKING CACHE 277
func (memo *Memo) Get(key string) (value interface{}, err error) {
memo.mu.Lock()
e:=memo.cache[key]
if e == nil {
// This is the first request for this key.
// This goroutine becomes responsible for computing
// the value and broadcasting the ready condition.
e=&entry{ready: make(chan struct{})}
memo.cache[key] = e
memo.mu.Unlock()
e.res.value, e.res.err = memo.f(key)
close(e.ready) // broadcast ready condition
}else {
// This is a repeat request for this key.
memo.mu.Unlock()
<-e.ready // wait for ready condition
}
return e.res.value, e.res.err
}
Acal l to Get nowinv olves acquir ingthe mut exlockthatguard s the cache map, looking inthe
mapfor a point er to an exist ing entry,allocat ingand ins ertinganew entry if non e was
found, thenreleasingthe lock. Ifthere was an exist ing entry,its value isnot necessarily ready
yetanot her goroutine cou ldstill becal lingthe slow function fso the cal linggoroutine
mu stwaitfor the entrys ‘‘re ady’’ condit ion beforeitreads the entrys result.Itdoesthisby
re adingavalue fro m the ready ch annel,since thisoperat ionblo cks until the channel isclos ed.
If there was noexist ing entry,thenbyins ertinganew ‘‘notready’’ entry into the map,the
currentgoroutine becom esresponsible for inv oking the slow function, updat ingthe entry,
andbro adc asting the readinessofthe new entry to any other goroutinesthatmig ht(by then)
be waiting for it.
No tice thatthe var iables e.res.value and e.res.err in the entry areshare d amon g
mu ltiplegoroutines. Thegoroutine thatcre atesthe entry sets their values, andother
goro utinesreadtheir values oncethe ‘‘re ady’’ condit ion has beenbro adc ast. Despit e being
accessedbymultiplegoroutines, nomut exlockisnecessary.The closingofthe ready ch annel
happ ens before anyother goroutine receivesthe bro adc astevent,sothe write to those var iables
in the rs t goro utine happ ens before they are read bysubsequentgoroutines. Thereisnodat a
race.
Ourcon cur rent, dup lic ate-sup pressing, non-blo cking cache iscomplete.
Theimp lementation of Memo ab ove usesamut extoguard a map variablethatisshare d by
each goroutine thatcal ls Get.Itsint erest ing tocontrastthisdesig n with analt ernat ive one in
whichthe map variableisconne d to a moni tor goroutine to whichcal lersof Get mu stsenda
mess age .
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
278 CHAPTER 9. CONCURRENCY WITH SHARED VARIABLES
Thedeclarat ions of Func, result,and entry remain as before:
// Func is the type of the function to memoize.
type Func func(key string) (interface{}, error)
// A result is the result of calling a Func.
type result struct {
value interface{}
err error
}
type entry struct {
res result
ready chan struct{} // closed when res is ready
}
Ho wever,the Memo type now con sists ofachannel, requests,through whichthe cal ler of Get
communic ates wit h themonit or goro utine.The elementtyp e of the channel isarequest.
Usingthisstr ucture, the cal ler of Get send s themonit or goro utine bot h thekey,thatis, the
argumenttothe memoize d func tion, andanother channel, response,overwhichthe result
shouldbesentbackwhenitbecom esavai lable.Thischannel will car ryonlyasinglevalue.
gopl.io/ch9/memo5
// A request is a message requesting that the Func be applied to key.
type request struct {
key string
response chan<- result // the client wants a single result
}
type Memo struct{ requests chan request }
// New returns a memoization of f. Clients must subsequently call Close.
func New(f Func) *Memo {
memo := &Memo{requests: make(chan request)}
go memo.server(f)
return memo
}
func (memo *Memo) Get(key string) (interface{}, error) {
response := make(chan result)
memo.requests <- request{key, response}
res := <-response
return res.value, res.err
}
func (memo *Memo) Close() { close(memo.requests) }
The Get method,above , createsarespons e ch annel,putsitinthe request, sends ittothe moni-
torgoroutine,thenimmediate lyreceivesfro m it.
The cache var iable isconne d to the monitorgoroutine (*Memo).server,shown below.The
monitorreads requests in a loop unt i l therequestchannel isclos edbythe Close method.For
each request, itcon sults the cache,cre ating and ins ertinganew entry if non e was found.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 9.7. EXAMPLE: CONCURRENT NON-BLOCKING CACHE 279
func (memo *Memo) server(f Func) {
cache := make(map[string]*entry)
for req := range memo.requests {
e:=cache[req.key]
if e == nil {
// This is the first request for this key.
e=&entry{ready: make(chan struct{})}
cache[req.key] = e
go e.call(f, req.key) // call f(key)
}
go e.deliver(req.response)
}
}
func (e *entry) call(f Func, key string) {
// Evaluate the function.
e.res.value, e.res.err = f(key)
// Broadcast the ready condition.
close(e.ready)
}
func (e *entry) deliver(response chan<- result) {
// Wait for the ready condition.
<-e.ready
// Send the result to the client.
response <- e.res
}
In a simi lar manner tothe mut ex-b ased version, the rs t re questfor a given key becom es
resp onsible for cal lingthe function f on thatkey,storing the resultinthe entry,and bro ad-
cast ing the readinessofthe entry by closingthe ready ch annel.Thisisdon e by
(*entry).call.
Asubsequentrequestfor the samekey nd s theexist ing entry in the map,waits for the result
to becom e re ady,and sends the resultthrough the respons e ch annel tothe clientgoroutine
that cal le d Get.Thisisdon e by (*entry).deliver.The call and deliver methodsmustbe
called intheir own goroutinestoens ure thatthe monitorgoroutine doesnot stoppro cessing
ne w re quests.
Thisexampleshows thatitspossibletobui ld many con cur rentstr uctures usingeit her ofthe
twoappro ach essh are d var iables andlocks, orcommunic atingsequential pro cesses
withoutexcessive complexity.
Itsnot always obv iou s whichappro ach ispreferable in a given situ ation,but itswor th
know ing how the y correspond.Som etimesswitchingfro m on e approach to the other can
make yourcodesimpler.
Exercis e 9.3: Extend the Func type and the (*Memo).Get method sothatcal lersmay provide
an opt ion al done ch annel through whichthe y cancancelthe operat ion(§8.9). Theresults ofa
cancel led Func call shouldnot becache d.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
280 CHAPTER 9. CONCURRENCY WITH SHARED VARIABLES
9.8. Goroutines and Threads
In the pre vious chapt erwesaid thatthe dif ference bet weengoroutinesand operat ingsystem
(OS) threads cou ldbeignored unt i l later. Alt hough the dif ferences bet weenthemare ess en-
tial lyquant itative , abig enough quant itative dif ference becom esaqualitative one,and soitis
with goroutinesand threads.The timehas now com e to distinguish them.
9.8.1. Gro wable Stacks
Each OSthreadhas a xe d-size blo ckofmemor y (often as large as2MB) for its sta ck,the wor k
area where itsaves the local variables offunctioncal lsthatare inprogressortemporar ily
su spende d whileanother functioniscal le d.This xe d-size stack issimultane ously too much
andtoo lit tle.A2MB stack wou ldbeahugewaste ofmemor y foralitt legoroutine,suchas
on e that merelywaits for a WaitGroup then clos es achannel.Itsnot uncommonfor a Go
prog ram to cre ate hundredsofthous and s of goroutinesatone time, whichwou ldbeimp ossi-
blewit h st acksthislarge . Ye tdespit e their size, xe d-size stacksare not always big enough for
themostcomplex anddeeply rec ursiveoffunctions.Chang ing the xe d size can improv e
sp ace efficiency and allow morethreads tobecre ate d, or it can enable moredeeply rec ursive
func tions,but it cannot dobot h.
In contrast, a goroutine startslifewit h asmall stack,typic ally 2KB.Agoro utinesstack,like
thestack ofanOSthread, holds the local variables ofactiveand suspended functioncal ls, but
un li keanOSthread, a goroutinesstack isnot xe d;itgrows andshr inks as needed.The size
limitfor a goroutine stack may beasmuchas1GB,ordersofmag nitude largerthanatypic al
xe d-size threadstack,thoug h of cours e fe w goro utinesuse thatmuch.
Exercis e 9.4: Cons truct a pip elinethatconne cts an arbit rar y numb erofgoroutineswit h ch an-
nel s.Whatisthe maximum numberofpip elinestagesyou can create wit houtrunningout of
memory?How lon g do es avalue taketotransitthe ent ire pip eline?
9.8.2. Goroutine Scheduling
OS threads are sch edu led bythe OSker nel.Every few millis econd s,ahardwaretimer inter-
ruptsthe pro cessor, whichcausesaker nel functioncal le d the sche dul er to beinv oke d.This
func tionsuspend s thecur rentlyexe cut ing threadand saves its reg istersinmemor y,looks over
thelistofthreads and decides whichone shouldrun next, restoresthatthreadsreg istersfro m
memory,thenresumes the exe cut ion of thatthread. Because OSthreads are sch edu led bythe
kernel,passingcontrol fro m on e thre adtoanother requires a full cont ext switch,thatis, sav ing
thestate ofone userthreadtomemor y,restoring the state ofanother,and updat ingthe
sche dulersdat a st ruc tures. Thisoperat ionisslow, due toits poorlocality and the numberof
memory accessesrequired,and has histor icallyonlygot ten wor seasthe numberofCPU cyc les
re quired toaccessmemor y hasincre ased.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 9.8. GOROUTINES AND THREADS 281
TheGoruntime containsits own sch edu ler thatusesatechnique known as m:n sch edu ling,
becaus e it multiplexes (orsch edu les) m goro utineson n OS threads.The job ofthe Go
sche duler isanalogou s to thatofthe ker nel sch edu ler,but it iscon cer ned onlywit h the
goro utinesofasingleGoprogram.
Un likethe operat ingsystemsthreadsch edu ler,the Gosch edu ler isnot inv oke d periodic ally
by a hardwaretimer,but imp licitlybycer tain Golangu agecon str ucts. For example, whena
goro utine cal ls time.Sleep or blo cks in a channel ormut exoperat ion, the sch edu ler putsitto
sleepand runsanother goroutine unt i l it istimetowakethe rs t on e up.Because itdoesnt
ne e d aswitch to ker nel context, res che dulingagoroutine ismuchche aperthanres che dulinga
thre ad.
Exercis e 9.5: Wr ite a program wit h twogoroutinesthatsendmessagesbackand for thover
twounbuf fered channel s in ping-p ong fashion. How manycommunic ations per secon d can
theprogram sustain?
9.8.3. GOMAXPROCS
TheGosch edu ler usesaparameter cal le d GOMAXPROCS to deter minehow many OS threads
maybeactivelyexe cut ing Gocodesimultane ously.Its default value isthe numberofCPUson
themachine, soonamachinewit h 8CPUs, the sch edu ler will sch edu leGocodeonupto8OS
thre ads atonce. (GOMAXPROCS is the n in m:n sche duling.)Goroutinesthatare sle epingor
blocke d in a communic ation do not need a thread at all.Goroutinesthatare blo cke d in I/O or
ot her systemcal lsorare cal lingnon-G o func tions,doneed anOSthread, but GOMAXPROCS
ne e d notaccountfor them.
Yo u canexplicitlycontrol thisparameter usingthe GOMAXPROCS enviro nment var iable orthe
runtime.GOMAXPROCS func tion. Wecan see the effec t of GOMAXPROCS on thislit tle program,
whichprints an end lessstreamofzeros andones:
for {
go fmt.Print(0)
fmt.Print(1)
}
$GOMAXPROCS=1 go run hacker-cliché.go
111111111111111111110000000000000000000011111...
$GOMAXPROCS=2 go run hacker-cliché.go
010101010101010101011001100101011010010100110...
In the rs t run, at mostone goroutine was exec ute d at a time. Initial ly, itwas the main
goro utine,whichprintsones. After a per iodoftime, the Gosch edu ler put it tosle ep andwoke
up the goroutine thatprintszeros, giv ing itatur n to run onthe OSthread. Inthe secon d run,
thereweretwo OSthreads avai lable,sobot h goro utinesran simultane ously,print ing dig its at
ab out the samerate. Wemuststressthatmanyfac tor s areinv olved ingoroutine sch edu ling,
andthe runtime iscon stant lyevo l ving, soyourresults may differfro m theonesabove .
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
282 CHAPTER 9. CONCURRENCY WITH SHARED VARIABLES
Exercis e 9.6: Me asure how the per for mance ofacompute-b oundparal lelprogram (see Exer-
cise 8.5) varies wit h GOMAXPROCS.Whatisthe opt imalvalue onyourcomputer? How many
CPUsdoesyourcomputerhave?
9.8.4. Goroutines HaveNoIdentity
In mostoperat ingsystems and programminglangu ages thatsup por t mu ltithre ading, the cur-
rent threadhas a dist inc t identity thatcan beeasi lyobt ained asanordinar y value,typic ally an
integerorpoint er. Thismakes iteasy tobui ld an abstrac tioncal le d thre ad-local storage,which
is ess ent ial lyaglobalmap keyed bythreadidentity,sothateachthreadcan store and ret rie ve
values indep endentofother threads.
Goro utineshavenonot ion of identity thatisaccessibletothe programmer.Thisisbydesig n,
since thread-lo cal storagetends tobeabu sed.For example, inaweb ser ver implemente d in a
language wit h thre ad-lo cal storage, itscommonfor manyfunctions tofind infor mat ionabout
theHTTPrequestonwhose beh alf the y arecur rentlywor kingbylooking inthatstorage.
Ho wever,justaswit h prog ramsthatrelyexcessive lyonglobalvar iables, thiscan lead toan
un healt hy ‘‘ac tionatadistance’’ in whichthe beh avior of a functionisnot deter mined byits
arguments alone,but bythe identity ofthe threadinwhichitruns. Con sequently, ifthe iden-
tity ofthe threadshouldchangesome wor ker threads are enliste d to help, saythefunction
misb ehavesmysteriou sly.
Go encourages a simpler sty leofprogramminginwhichparametersthataffec t thebeh avior of
afunctionare explicit. Not onlydoesthismakeprogramseasier toread, but it lets usfre ely
assig n su btasksofagiven functiontomanydif ferentgoroutineswit houtwor rying about their
identity.
Yo u ve now learned about all the langu agefeaturesyou need for writing Goprograms. Inthe
next two chapt ers,well stepbacktolookatsom e of the prac tices andtools thatsup por t
prog ramminginthe large:how tostr uctureapro jec t as a set ofpackages, andhow toobt ain,
build, test,benchmark,profile,document, andshare those packages.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
10
Packages and the GoTool
Amodest-size program today might contain 10,000 functions.Yet its author need thin k ab out
on lyafew ofthemand desig n even fewer,because the vastmaj ority werewritt enbyothers
andmade availablefor reuse through pack ages.
Go com eswit h ov er100 stand ard packages thatprovide the found ation s formostapp lic ations.
TheGocommunity,athr iving ecosystemofpackagedesig n, sh aring , reus e,and imp rov ement,
haspublishe d many more, and you can nd a searc hable index ofthemat http://godoc.org.
In thischapt er, well showhow touse exist ing packages andcre ate new ones.
Go als o comeswit h the go to ol, a sop histicate d butsimple-to-use command for manag ing
worksp aces ofGopackages. Since the beg inningofthe book, weve beenshowing how touse
the go to oltodow nlo ad,bui ld,and run exampleprograms. Inthischapt er, well lookatthe
to olsunderly ing con cepts andtourmoreofits cap abilit ies, whichinclude print ing documen-
tation and quer yingmet adat a ab out the packages in the wor ksp ace.Inthe next chapt erwell
explore its testing features.
10.1. Introduction
Thepur pos e of any packagesystemistomakethe desig n andmaintenanceoflarge programs
prac tic al by gro uping rel ate d fe aturestoget her into units thatcan beeasi lyunderstood and
ch ange d,indep endentofthe other packages ofthe program. This mo dul arity al lows packages
to beshare d andreusedbydif ferentpro jec ts, dist ribut edwit hin an organizat ion, ormade
avai lable tothe wider wor ld.
Each packagedefinesadistinc t name space thatenclos es itsidentifiers. Eachnameisass oci-
ated wit h apar tic ularpackage, letting uscho ose short,cle ar namesfor the typ es, func tions,
andsoonthatweuse mostoften, wit houtcre ating conflic ts with other par ts of the program.
283
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
284 CHAPTER 10. PACKAGES AND THE GO TOOL
Packages also provide encapsul ati on by control lingwhichnames arevisible orexp orted
outside the package. Restr ictingthe visibilit y of packagemembers hides the helperfunctions
andtyp es behindthe packagesAPI,allow ing the packagemaintainer tochange the imp lemen-
tation wit h confidence thatnocodeoutside the packagewill beaffec ted.Restr ictingvisibilit y
also hides variables sothatclients can accessand updatethemonlythrough exp orted func-
tion s that preserveint ernal invar iants orenforce mut ual exclusion in a con cur rentprogram.
Wh enwechange a file,wemustrecompi lethe filespackageand pot ent ial lyall the packages
that dependonit. Gocompi lat ionisnot ably fasterthanmostother comp ile d languages, even
when bui ldingfro m scratch.There are three main reasons for the compi lersspeed.First,all
importsmustbeexplicitlyliste d at the beg inningofeachsource file,sothe compi ler does not
have toreadand pro cessanent ire file todeter mineits dependencies. Secon d,the dep enden-
cies ofapackagefor m adirec ted acyclic graph, andbecause there are nocyc les, packages can
be compi led sep aratelyand perhaps in paral lel. Final ly, the obj e ctfile for a compi led Gopack-
agerecords exp ort infor mat ionnot justfor the packageits elf,but for itsdep endencies too.
Wh encompi lingapackage, the compi ler mustreadone obj e ctfile for eachimp ort but need
notlookbeyon d thes e files.
10.2. Import Paths
Each packageisidentied byaunique str ing cal le d its import pat h.Imp ort pat hsare the
st rings thatapp ear in import de clarat ions.
import (
"fmt"
"math/rand"
"encoding/json"
"golang.org/x/net/html"
"github.com/go-sql-driver/mysql"
)
As wemention edinSec tion 2.6.1, the Golangu agespecification doesntdefine the meaningof
thes e st rings orhow todeter mineapackagesimp ort pat h, butleavesthese issues tothe tools.
In thischapt er, well takeadet aile d lo okathow the go to olint erprets them, since thatswhat
themaj ority ofGoprogrammersuse for bui lding, testing , andsoon. Other tools doexist,
though.For example, GoprogrammersusingGooglesint ernal multi-l angu agebui ld system
fo llowdif ferentrules for namingand locat ingpackages, specif yingtests, andsoon, thatmore
clos ely match the convent ion s of thatsystem.
Fo r packages you int end toshare orpublish,imp ort pat hsshouldbegloballyunique.Toavoid
conflic ts, theimp ort pat hsofall packages other thanthose fro m thestand ard librar y should
st art wit h theInt ernet domain nameofthe organizat ionthatownsorhosts the package; this
also makes itpossibletofind packages. For example, the declarat ionabove imp ortsanHTML
pars ermaintained bythe Goteamand a popu lar third-p arty MySQLdat abas e dr iver.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 10.4. IMPORTDECLARATIONS 285
10.3. The PackageDeclaration
A package de clarat ionisrequired atthe start ofevery Gosource file.Its main pur pos e is to
deter minethe defau ltidentifier for thatpackage(called the pack age nam e)whenitisimp orted
by another package.
Fo r example, every file ofthe math/rand packagestartswit h package rand,sowhenyou
import thispackage, you can accessits members as rand.Int, rand.Float64,and soon.
package main
import (
"fmt"
"math/rand"
)
func main() {
fmt.Println(rand.Int())
}
Conv ent ion ally, the packagenameisthe lastseg mentofthe imp ort pat h, andasaresult, two
packages may havethe samenameeventhoug h their import pat hsnecessarily dif fer.For
example, the packages whose imp ort pat hsare math/rand and crypto/rand both havethe
name rand.Well see how touse bot h in the sameprogram in a mom ent.
Thereare three maj orexception s to the ‘‘last seg ment’’ conv ent ion.The rs t is thatapackage
definingacommand (an exec utableGoprogram) always has the name main,regardlessofthe
packagesimp ort pat h. Thisisasig nal to go build (§10.7.3) thatitmustinv oke the lin ker to
make anexe cut able file.
Thesecon d exception isthatsom e files in the direc tor y mayhavethe suffix _test on their
packagenameifthe file nameend s with _test.go.Suchadirec tor y maydefine two packages:
theusu alone,plu s anot her one cal le d an exter nal testpackage.The _test suffixsig nalsto
go test that itmustbui ld both packages, anditindic ates which files belon g to eachpackage.
External testpackages areusedtoavoid cyc les in the imp ort grapharisingfro m dep endencies
of the test; the y arecov ere d in moredet ail in Sec tion 11.2.4.
Thethirdexception isthatsom e to ols for dep endency managementapp end versionnumber
suffixestopackageimp ort pat hs, suchas "gopkg.in/yaml.v2".The packagenameexc ludes
thesuffix, so in thiscas e it wou ldbejust yaml.
10.4. Import Declarations
AGosourcefile may cont ain zeroormore import de clarat ions immediate lyafter the package
de clarat ionand beforethe rs t non-import declarat ion. Eachimp ort declarat ionmay specif y
theimp ort pat h of a singlepackage, ormultiplepackages in a parenthesize d list.The two
formsbelow are equivalentbut the secon d form ismorecommon.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
286 CHAPTER 10. PACKAGES AND THE GO TOOL
import "fmt"
import "os"
import (
"fmt"
"os"
)
Imported packages may begro upedbyint roducingblank lines; suchgro upingsusu allyindi-
cate dif ferentdom ains. Theorder isnot sig nificant,but byconvent ion the lines ofeachgro up
aresor ted alp hab etic ally.(Both gofmt and goimports wi l l groupand sor t foryou .)
import (
"fmt"
"html/template"
"os"
"golang.org/x/net/html"
"golang.org/x/net/ipv4"
)
If weneed toimp ort two packages whose names arethe same, like math/rand and
crypto/rand,int o athirdpackage, the imp ort declarat ionmustspecif y an alternat ive name
for at least one ofthemtoavoid a conflic t. Thisiscal le d a renami n g import.
import (
"crypto/rand"
mrand "math/rand" // alternative name mrand avoids conflict
)
Thealt ernat ive nameaffec tsonlythe imp orting file.Other files, evenonesinthe samepack-
age, may import the packageusingits default name, oradif ferentname.
Arenamingimp ort may beusefulevenwhenthere isnoconflic t. If the nameofthe imp orted
packageisunw ieldy,asissom etimesthe cas e foraut omat ical lygenerated code, anabbre viate d
name may bemoreconvenient. Thesameshort nameshouldbeusedcon sistent lytoavoid
conf usion. Cho osinganalt ernat ive namecan helpavoid conflic ts with commonlocal variable
names. For example, inafile wit h many local variables named path,wemig htimp ort the
st and ard "path" packageas pathpkg.
Each imp ort declarat ionest ablishesadep endency fro m thecur rentpackagetothe imp orted
package. The go build to olrep orts an error ifthese dep endencies for m acyc le.
10.5. Blank Imports
It isanerror toimp ort a packageint o afile but not refer tothe nameitdefineswit hin that file.
Ho wever,onocc asionwemustimp ort a packagemerelyfor the side effe cts ofdoing so: evalu-
at ionofthe initializer expressions ofits package-le vel var iables andexe cut ion of its init func-
tion s (§2.6.2). Tosup pressthe ‘‘unus edimp ort’’ er ror wewou ldother wis e encounter, wemust
us e arenamingimp ort inwhichthe alt ernat ive nameis _,the blank identifier.Asusu al, the
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 10.5. BLANK IMPORTS 287
bl ank identifier can never bereference d.
import _ "image/png" // register PNG decoder
Thisisknown asabl ank imp ort.Itismostoften usedtoimp lementacompi le-t ime
me chanism where bythe main program can enable opt ion alfeaturesbyblank-importingaddi-
tion alpackages. First well see how touse it, thenwell see how itwor ks.
Thestand ard librar ys image packageexp ortsaDecode func tionthatreads bytes fro m an
io.Reader, figures out whichimage for mat was usedtoencodethe dat a, invo kes the
appropriatedecoder,thenretur nsthe resulting image.Image.Using image.Decode,itseasy
to bui ld asimpleimage converter thatreads animage inone for mat andwritesitout in
anot her :
gopl.io/ch10/jpeg
// The jpeg command reads a PNG image from the standard input
// and writes it as a JPEG image to the standard output.
package main
import (
"fmt"
"image"
"image/jpeg"
_"image/png" // register PNG decoder
"io"
"os"
)
func main() {
if err := toJPEG(os.Stdin, os.Stdout); err != nil {
fmt.Fprintf(os.Stderr, "jpeg: %v\n", err)
os.Exit(1)
}
}
func toJPEG(in io.Reader, out io.Writer) error {
img, kind, err := image.Decode(in)
if err != nil {
return err
}
fmt.Fprintln(os.Stderr, "Input format =", kind)
return jpeg.Encode(out, img, &jpeg.Options{Quality: 95})
}
If wefeed the out put of gopl.io/ch3/mandelbrot (§3.3) tothe converter program, itdetec ts
thePNG inp utfor mat andwritesaJPEGversionofFigure3.3.
$gobuild gopl.io/ch3/mandelbrot
$gobuild gopl.io/ch10/jpeg
$./mandelbrot | ./jpeg >mandelbrot.jpg
Input format = png
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
288 CHAPTER 10. PACKAGES AND THE GO TOOL
No tice the blank imp ort of image/png.Wit houtthatline, the program comp iles andlin ks as
usualbut can nolon g errecog nize or decodeinp ut in PNG for mat:
$gobuild gopl.io/ch10/jpeg
$./mandelbrot | ./jpeg >mandelbrot.jpg
jpeg: image: unknown format
Hereshow itwor ks. Thestand ard librar y prov ides deco dersfor GIF,PNG,and JPEG, and
us ers may provide others, but tokeepexe cut ables smal l,decodersare not include d in an appli-
cation unlessexplicitlyrequeste d. The image.Decode func tioncon sults a table ofsup por ted
formats. Eachent ryinthe table specifies fourthings: the nameofthe for mat; a str ing thatisa
prefixofall imagesencoded thisway,usedtodetec t theencoding; a function Decode that
de codes an enco dedimage;and another function DecodeConfig that deco des on lythe image
metadata,suchasits size andcolor space.Anent ryisadde d to the table bycal ling
image.RegisterFormat,typic ally fro m within the packageinitializer ofthe sup por tingpack-
agefor eachfor mat, like thisone in image/png:
package png // image/png
func Decode(r io.Reader) (image.Image, error)
func DecodeConfig(r io.Reader) (image.Config, error)
func init() {
const pngHeader = "\x89PNG\r\n\x1a\n"
image.RegisterFormat("png", pngHeader, Decode, DecodeConfig)
}
Theeffec t is thatanapp lic ationneed onlyblank-import the packagefor the for mat itneedsto
make the image.Decode func tionabletodecodeit.
The database/sql packageusesasimi lar mechanism tolet users ins tal l ju stthe dat abas e
dr ivers the y ne e d.For example:
import (
"database/mysql"
_"github.com/lib/pq" // enable support for Postgres
_"github.com/go-sql-driver/mysql" // enable support for MySQL
)
db, err = sql.Open("postgres", dbname) // OK
db, err = sql.Open("mysql", dbname) // OK
db, err = sql.Open("sqlite3", dbname) // returns error:
unknown driver "sqlite3"
Exercis e 10.1: Extend the jpeg prog ram so thatitconvertsany sup por ted inp utfor mat toany
output for mat, using image.Decode to detec t theinp utfor mat andaflag toselec t theout put
format.
Exercis e 10.2: Dene a gener ic archive file-readingfunctioncap able ofreadingZIP files
(archive/zip)and POSIX tar files (archive/tar). Use a reg ist rat ionmechanism similarto
theone des crib edabove sothatsup por t foreach file for mat can beplugged inusingblank
imports.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 10.6. PACKAGES AND NAMING 289
10.6. Packages and Naming
In thissec tion,well offer som e advice onhow tofol low Gosdistinc tiveconvent ion s fornam-
ingpackages andtheir members.
Wh encre ating a package, keepits nameshort,but not soshort astobecrypt ic. Themost
frequentlyusedpackages in the stand ard librar y arenamed bufio, bytes, flag, fmt, http, io,
json, os, sort, sync,and time.
Be des crip tiveand unambiguous where possible. For example, dontnameautilit y package
util when a namesuchas imageutil or ioutil is specificyet still con cis e.Avoid cho osing
packagenames thatare commonlyusedfor rel ate d lo cal variables, oryou may comp elthe
packagesclients touse renamingimp orts, as wit h the path package.
Packagenames usu allytakethe singu lar for m.The stand ard packages bytes, errors,and
strings us e theplural toavoid hidingthe cor respondingpre declare d typesand, in the cas e of
go/types,toavoid conflic t with a key word.
Av oid packagenames thatalready haveother connot ation s.For example, weoriginallyused
thename temp forthe temperatureconversionpackageinSec tion 2.5, but thatdidntlastlon g.
It was a ter r ibleide a becaus e ‘‘temp’’ is analmostunivers alsynonym for ‘‘temp orar y.’’ We went
thro ugh a brief per iodwit h thename temperature,but thatwas too lon g anddidntsay what
thepackagedid.Inthe end,itbecame tempconv,whichisshorter andparal lelwit h strconv.
No w letstur n to the namingofpackagemembers.Since eachreference toamemberof
anot her packageusesaqualified identifier suchas fmt.Println,the burden ofdes cribingthe
packagememberisbor neequ allybythe packagenameand the memb ername. Weneed not
ment ion the con ceptoffor matting in Println becaus e thepackagename fmt do es that
already.Whendesig ningapackage, con sider how the two par ts of a qualified identifier wor k
together,not the memb ernamealone.Hereare som e ch arac ter ist icexamples:
bytes.Equal flag.Int http.Get json.Marshal
We c an identify som e common namingpatterns. The strings packageprovides a numberof
indep endentfunctions for manipu lat ingstr ings:
package strings
func Index(needle, haystack string) int
type Replacer struct{ /* ... */ }
func NewReplacer(oldnew ...string) *Replacer
type Reader struct{ /* ... */ }
func NewReader(s string) *Reader
Theword string do es notapp ear in anyoftheir names. Clients refer tothemas
strings.Index, strings.Replacer,and soon.
Ot her packages thatwemig htdes crib e as single-ty pepackages,suchas html/template and
math/rand,exp oseone princip aldat a type plu s itsmet hods, andoften a New func tiontocre-
ateins tances.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
290 CHAPTER 10. PACKAGES AND THE GO TOOL
package rand // "math/rand"
type Rand struct{ /* ... */ }
func New(source Source) *Rand
Thiscan lead torep etit ion,asin template.Template or rand.Rand,whichiswhy the names
of these kinds ofpackages areoften especi ally short.
At the other extreme,there are packages like net/http that havealot ofnames wit houta lot
of str ucture, because the y perfor m acomplic ated task. Despit e having overtwent y typesand
many morefunctions,the packagesmostimp ortantmembers havethe simplestnames: Get,
Post, Handle, Error, Client, Server.
10.7. The GoTool
Therestofthischapt ercon cer nsthe go to ol, whichisusedfor dow nlo ading, quer ying,
formatting , building, testing , andins tal lingpackages ofGocode.
The go to olcom binesthe featuresofadiverse setoftools int o on e commandset. Itisapack-
agemanager (analogou s to apt or rpm)thatanswers quer ies ab out itsinv ent ory ofpackages,
comp utestheir dependencies, anddow nlo adsthemfro m remote version-cont rol systems.Itis
abui ld systemthatcomputesfile dependencies andinv okescompi lers, assemb lers, andlin k-
ers, although itisint ent ion allylesscompletethanthe stand ard Unix make.And itisatest
dr iver, aswewill see inChapt er11.
Itscommand-lineint erface usesthe ‘‘Sw iss armyknife’’ st yle,wit h ov eradozen sub com-
mand s,som e of whichwehavealready seen, like get, run, build,and fmt.You can run
go help to see the index ofits bui lt-in document ation,but for reference,weve liste d themost
common lyusedcommand s below:
$go
...
build compile packages and dependencies
clean remove object files
doc show documentation for package or symbol
env print Go environment information
fmt run gofmt on package sources
get download and install packages and dependencies
install compile and install packages and dependencies
list list packages
run compile and run Go program
test test packages
version print Go version
vet run go tool vet on packages
Use "go help [command]" for more information about a command.
...
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 10.7. THE GO TOOL 291
To keepthe need for configurat iontoa minimum, the go to olrelies heavi lyonconvent ion s.
Fo r example, given the nameofaGo source file,the toolcan nd its enclosingpackage,
becaus e each direc tor y cont ainsasinglepackageand the imp ort pat h of a packagecor-
resp ond s to the direc tor y hierarchyinthe wor ksp ace.Given the imp ort pat h of a package, the
to olcan nd the cor respondingdirec tor y in whichitstoresobj e ctfiles. Itcan also nd the
URL ofthe ser ver thathosts the sourcecoderep ository.
10.7.1. WorkspaceOrganization
Theonlyconfigurat ionmostusers everneed isthe GOPATH enviro nment var iable,whichspeci-
fies the rootofthe wor ksp ace.Whenswitchingtoadif ferentwor ksp ace,users updatethe
value of GOPATH.For ins tance,weset GOPATH to $HOME/gobook whilewor kingonthisbook:
$export GOPATH=$HOME/gobook
$goget gopl.io/...
Af ter you dow nlo ad al l theprogramsfor thisbookusingthe command above , your wor ksp ace
wi l l cont ain a hierarchylikethisone:
GOPATH/
src/
gopl.io/
.git/
ch1/
helloworld/
main.go
dup/
main.go
...
golang.org/x/net/
.git/
html/
parse.go
node.go
...
bin/
helloworld
dup
pkg/
darwin_amd64/
...
GOPATH hasthree sub direc tor ies. The src su bdirec tor y holdssourcecode. Eachpackage
resides in a direc tor y whos e name rel ative to $GOPATH/src is the packagesimp ort pat h, such
as gopl.io/ch1/helloworld.Obs erve thatasingle GOPATH worksp ace cont ainsmultiplever-
sion-cont rol rep ositories beneath src,suchas gopl.io or golang.org.The pkg su bdirec tor y
is where the bui ld to ols store compi led packages, andthe bin su bdirec tor y holdsexe cut able
prog ramslike helloworld.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
292 CHAPTER 10. PACKAGES AND THE GO TOOL
Asecon d enviro nment var iable, GOROOT,specifies the rootdirec tor y of the Godistr ibution,
whichprovides allthe packages ofthe stand ard librar y.The direc tor y st ruc turebeneath
GOROOT resemb les thatof GOPATH,so, for example, the sourcefiles ofthe fmt packagereside in
the $GOROOT/src/fmt direc tor y.Users never need toset GOROOT since,bydefau lt, the go to ol
wi l l us e thelocat ionwhere itwas inst alled.
The go env commandprintsthe effec tive values ofthe env iro nment var iables relevanttothe
to olchain, includingthe defau ltvalues for the missingones. GOOS sp ecifies the targetoperat-
ingsystem(forexample, android, linux, darwin,or windows)and GOARCH sp ecifies the target
processorarc hit ecture, suchas amd64, 386,or arm.Alt hough GOPATH is the onlyvar iable you
mu stset, the othersocc asionallyapp ear in our explanation s.
$goenv
GOPATH="/home/gopher/gobook"
GOROOT="/usr/local/go"
GOARCH="amd64"
GOOS="darwin"
...
10.7.2. Downloading Packages
Wh enusingthe go to ol, a packagesimp ort pat h indic ates not onlywhere tofind itinthe local
worksp ace,but where tofind itonthe Int ernet sothat go get canret rie veand updateit.
The go get commandcan down loadasinglepackageoranent ire subtree orrep ository using
the ... notation, as in the pre vious sec tion.The toolals o comp utesand dow nlo adsall the
dep endencies ofthe initial packages, whichiswhy the golang.org/x/net/html package
appeared inthe wor ksp ace in the pre vious example.
Once go get hasdow nlo ade d thepackages, itbui ldsthemand then install s thelibrar ies and
commands.Well lookatthe det ails inthe next sec tion,but anexamplewill showhow
st raig htfor wardthe pro cessis. The rs t commandbelow gets the golint to ol, whichche cks
forcommonsty lepro blemsinGosourcecode. The secon d commandruns golint on
gopl.io/ch2/popcount from Sec tion 2.6.2. Ithelpf ullyrep ortsthatwehavefor g ott ento
wr ite a doc comment for the package:
$goget github.com/golang/lint/golint
$$GOPATH/bin/golint gopl.io/ch2/popcount
src/gopl.io/ch2/popcount/main.go:1:1:
package comment should be of the form "Package popcount ..."
The go get commandhas sup por t forpopu lar code-hosting sit eslikeGitHu b,Bitbucket, and
Launchpadand can makethe appro priaterequests totheir version-cont rol systems.For less
we ll-k now n sites, you may havetoindic atewhichversion-cont rol pro tocol touse inthe
import pat h, such asGit or Mercurial.Run go help importpath forthe det ails.
Thedirec tor ies that go get createsare trueclients ofthe remot e repository,not justcopies of
thefiles, soyou can use version-cont rol command s to see a dif f of local edits youve made orto
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 10.7. THE GO TOOL 293
up datetoadif ferentrevision. For example, the golang.org/x/net direc tor y is a Git client:
$cd$GOPATH/src/golang.org/x/net
$git remote -v
origin https://go.googlesource.com/net (fetch)
origin https://go.googlesource.com/net (push)
No tice thatthe app arent dom ain name in the packagesimp ort pat h, golang.org,dif fersfro m
theactualdom ain nameofthe Git ser ver, go.googlesource.com.Thisisa feature ofthe go
to olthatlets packages use a customdom ain nameintheir import pat h whilebeinghoste d by a
genericser vice suchas googlesource.com or github.com.HTMLpages beneath
https://golang.org/x/net/html include the metadata shown below,whichredirec tsthe go
to oltothe Git rep ository atthe actualhosting sit e:
$gobuild gopl.io/ch1/fetch
$./fetch https://golang.org/x/net/html | grep go-import
<meta name="go-import"
content="golang.org/x/net git https://go.googlesource.com/net">
If you specif y the -u flag , go get wi l l ensure thatall packages itvisits, includingdep endencies,
areupdated totheir latestversionbeforebeingbui ltand ins tal le d.Wit houtthatflag , packages
that already exist local lywill not beupdated.
The go get -u commandgeneral lyret rie ves the latestversionofeachpackage, whichiscon-
venientwhenyoure getting started but may beinappro priatefor deploye d proj e cts, where
precis e cont rol ofdep endencies iscriticalfor release hyg iene. The usu alsolut ion tothis
problem isto vend or thecode, thatis, tomakeapersistentlocal copy ofall the necessary
dep endencies, andtoupdatethiscopycaref ullyand delib erately. Prior toGo1.5, thisrequired
ch ang ing those packages import pat hs, soour copy of golang.org/x/net/html would
become gopl.io/vendor/golang.org/x/net/html.Morerecentversions ofthe go to ol
supp ort vendoring direc tly,thoug h we donthavespace toshowthe det ails here. See Vend or
Dire ctor ies in the out put of the go help gopath command.
Exercis e 10.3: Using fetch http://gopl.io/ch1/helloworld?go-get=1,find out which
service hosts the codesamples for thisbook. (HTTP requests fro m go get include the go-get
parameter sothatser verscan dist inguish themfro m ordinar y browserrequests.)
10.7.3. Building Packages
The go build commandcompi les eachargumentpackage. Ifthe packageisalibrar y,the
resu ltisdis carde d; this merelyche cks thatthe packageisfre e of compi leerror s.Ifthe package
is name d main, go build invo kes the lin ker tocre ate anexe cut ableinthe cur rentdirec tor y;
thenameofthe exe cut ableistaken fro m thelastseg mentofthe packagesimp ort pat h.
Sinceeachdirec tor y cont ainsone package, eachexe cut ableprogram, or comm and in Unix ter-
minolog y,requires its own direc tor y.These direc tor ies aresom etimeschi ldren ofadirec tor y
name d cmd,suchasthe golang.org/x/tools/cmd/godoc commandwhichser ves Gopack-
agedocumentation through a web int erface (§10.7.4).
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
294 CHAPTER 10. PACKAGES AND THE GO TOOL
Packages may bespecified bytheir import pat hs, as wesaw abov e,orbyarel ative direc tor y
name,whichmuststart wit h a . or .. segmentevenifthiswou ldnot ordinar ily berequired.
If noargumentisprovide d,the cur rentdirec tor y is assume d.Thu s thefol low ing command s
buildthe samepackage, thoug h each writesthe exe cut abletothe direc tor y in which go build
is run:
$cd$GOPATH/src/gopl.io/ch1/helloworld
$gobuild
and:
$cd
anywhere
$gobuild gopl.io/ch1/helloworld
and:
$cd$GOPATH
$gobuild ./src/gopl.io/ch1/helloworld
butnot:
$cd$GOPATH
$gobuild src/gopl.io/ch1/helloworld
Error: cannot find package "src/gopl.io/ch1/helloworld".
Packages may also bespecified asalistoffile names, thoug h this tends tobeusedonlyfor
smal l prog ramsand one-off exp eriments. Ifthe packagenameis main,the exe cut ablename
comesfro m thebas enameofthe rs t .go file.
$cat quoteargs.go
package main
import (
"fmt"
"os"
)
func main() {
fmt.Printf("%q\n", os.Args[1:])
}
$gobuild quoteargs.go
$./quoteargs one "two three" four\ five
["one" "two three" "four five"]
Partic ularlyfor throwaway programslikethisone,wewanttorun the exe cut ableassoonas
weve bui ltit. The go run commandcom binesthese two steps:
$gorun quoteargs.go one "two three" four\ five
["one" "two three" "four five"]
The rs t argumentthatdoesntend in .go is assume d to bethe beg inningofthe listofargu-
mentstothe Goexe cut able.
By defau lt, the go build commandbui ldsthe requeste d packageand all its dependencies, then
throws away allthe compi led codeexceptthe nalexe cut able, ifany.Bot h thedep endency
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 10.7. THE GO TOOL 295
analysisand the compi lat ionare sur prisinglyfast, but aspro jec tsgrowtodozensofpackages
andhundredsofthous and s of lines ofcode, the timetorecompi ledep endencies can becom e
notice able,pot ent ial lyseveral secon ds, evenwhenthose dep endencies haventchange d at all.
The go install commandisver y simi lar to go build,exceptthatitsaves the compi led code
foreachpackageand command ins teadofthrow ing itaway.Compi led packages aresaved
bene ath the $GOPATH/pkg direc tor y correspondingtothe src direc tor y in whichthe source
resides, andcommand exe cut ables aresaved inthe $GOPATH/bin direc tor y.(Many users put
$GOPATH/bin on their exec utablesearc h path.) Thereafter, go build and go install do not
runthe compi ler for those packages andcommand s if the y have not change d,mak ingsubse-
quentbui ldsmuchfaster. For convenience, go build -i inst allsthe packages thatare dep en-
dencies ofthe bui ld target.
Sincecompi led packages varybyplatfor m andarc hit ecture, go install savesthembeneath a
su bdirec tor y whos e name incor porates the values ofthe GOOS and GOARCH enviro nment var i-
ables. For example, onaMac the golang.org/x/net/html packageiscompi led and ins tal le d
in the file golang.org/x/net/html.a under $GOPATH/pkg/darwin_amd64.
It isstraig htfor wardto cross-compi le aGoprogram, thatis, tobui ld an exec utableint ended for
adif ferentoperat ingsystemorCPU.Justset the GOOS or GOARCH var iables dur ingthe bui ld.
The cross prog ram pr intsthe operat ingsystemand arc hit ecturefor whichitwas bui lt:
gopl.io/ch10/cross
func main() {
fmt.Println(runtime.GOOS, runtime.GOARCH)
}
Thefol low ing command s produce 64-bit and 32-bit exe cut ables respectively:
$gobuild gopl.io/ch10/cross
$./cross
darwin amd64
$GOARCH=386 go build gopl.io/ch10/cross
$./cross
darwin 386
Some packages may need tocompi ledif ferentversions ofthe codefor cer tain platfor msor
processors,todealwit h low-le vel por tabilit y issues ortoprovide opt imize d versions of
importantroutines, for ins tance.Ifafile nameincludes an operat ingsystemorpro cessor
archit ecturenamelike net_linux.go or asm_amd64.s,thenthe go to olwill compi lethe file
on lywhenbui ldingfor thattarget. Speci al comments cal le d buil d tags give morefine-g rained
cont rol . Fo r example, ifafile cont ainsthiscomment:
// +build linux darwin
before the packagedeclarat ion(andits doccomment), go build wi l l comp ile itonlywhen
buildingfor Linux orMac OS X, andthiscomment says never tocompi lethe file:
// +build ignore
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
296 CHAPTER 10. PACKAGES AND THE GO TOOL
Fo r more det ails,see the Bu ild Const rai nts sectionofthe go/build packagesdocumentation:
$godoc go/build
10.7.4. Documenting Packages
Go sty lestron gly encourages good documentation of packageAPIs. Eachdeclarat ionofan
exp orted packagememberand the packagedeclarat ionits elf shouldbeimmediate lypre ceded
by a comment explainingits pur pos e andusage .
Go doc comments arealways comp letesentences, andthe rs t sent enceisusu allyasummar y
that startswit h thenamebeingdeclare d.Functionparametersand other identifiersare men-
tion edwit houtquotation or marku p.For example, heresthe doc comment for fmt.Fprintf:
// Fprintf formats according to a format specifier and writes to w.
// It returns the number of bytes written and any write error encountered.
func Fprintf(w io.Writer, format string, a ...interface{}) (int, error)
Thedet ails of Fprintfsfor matting are explained inadoc comment ass oci ated wit h the fmt
packageits elf.Acommentimmediate lypre cedingapackage de clarat ioniscon sidered the
do c commentfor the packageasawhole.There mustbeonlyone,thoug h it may appear in
any file.Lon g erpackagecommentsmay warrantafile oftheir own; fmtsisover300 lines.
This file isusu allycal le d doc.go.
Go o d do cumentation need not beextensive , anddocumentation isnosubst itute for simplicity.
In deed,Gosconvent ion s favor brevit y andsimplicity indocumentation as in all things, since
do cumentation,likecode, requires maintenancetoo.Manydeclarat ions can beexplained in
on e we ll-worde d sent ence, and ifthe beh avior istruly obv iou s,nocomment isneeded.
Throughoutthe book, as space per mits, weve pre ceded manydeclarat ions bydoc comments,
butyou will nd betterexamples as you brows e thestand ard librar y.Two tools can helpyou
do that.
The go doc to olprintsthe declarat ionand doc comment ofthe ent ity specified onthe com-
mand line, whichmay beapackage:
$godoc time
package time // import "time"
Package time provides functionality for measuring and displaying time.
const Nanosecond Duration = 1 ...
func After(d Duration) <-chan Time
func Sleep(d Duration)
func Since(t Time) Duration
func Now() Time
type Duration int64
type Time struct { ... }
...
many more
...
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 10.7. THE GO TOOL 297
or a packagemember:
$godoc time.Since
func Since(t Time) Duration
Since returns the time elapsed since t.
It is shorthand for time.Now().Sub(t).
or a met hod:
$godoc time.Duration.Seconds
func (d Duration) Seconds() float64
Seconds returns the duration as a floating-point number of seconds.
Thetooldoesnot need completeimp ort pat hsorcor rec t identifier cas e.Thiscommand prints
thedocumentation of (*json.Decoder).Decode from the encoding/json package:
$godoc json.decode
func (dec *Decoder) Decode(v interface{}) error
Decode reads the next JSON-encoded value from its input and stores
it in the value pointed to by v.
Thesecon d to ol, confusinglynamed godoc,ser ves cross-lin ked HTMLpages thatprovide the
same infor mat ionas go doc andmuchmore. The godoc server at https://golang.org/pkg
covers the stand ard librar y.Figure10.1 shows the documentation for the time package, and
in Sec tion 11.6 well see godocsint erac tivedispl ayofexampleprograms. The godoc server at
https://godoc.org hasasearc hable index ofthous and s of open-s ource packages.
Yo u canals o runanins tance of godoc in yourwor ksp ace if you wanttobrows e your own
packages. Visit http://localhost:8000/pkg in yourbrows er whilerunningthiscommand:
$godoc -http :8000
Its -analysis=type and -analysis=pointer flags augment the documentation and the
source code wit h theresults ofadvance d st aticanalysis.
10.7.5. Int ernal Packages
Thepackageisthe most imp ortantmechanism for enc apsulat ioninGoprograms. Unex-
ported identifiersare visible onlywit hin the samepackage, and exp orted identifiersare visible
to the wor ld.
Sometimes, thoug h, amidd le ground wou ldbehelpf ul, a way todefine identifiersthatare visi-
bletoasmall set oftrusted packages, but not toeveryon e.For example, whenwere bre aking
up a large packageint o more manageablepar ts, we may not wanttorevealthe int erfaces
betweenthose par ts to other packages. Or we may wanttoshare utilit y func tions across
several packages ofapro jec t withoutexp osingthemmorewidely.Orperhaps wejustwantto
exp erimentwit h anew packagewit houtpremature lycommitt ing toits API, byput tingit ‘‘on
prob ation’’ with a limite d setofclients.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
298 CHAPTER 10. PACKAGES AND THE GO TOOL
Figure 10.1. The time packagein godoc.
To addressthese needs, the go build to oltre ats a packagespeci ally ifits import pat h cont ains
apat h segmentnamed internal.Suchpackages arecal le d internal packages.Anint ernal
packagemay beimp orted onlybyanother packagethatisinside the tre e ro ote d at the parent
of the internal direc tor y.For example, given the packages below, net/http/inter-
nal/chunked canbeimp orted fro m net/http/httputil or net/http,but not fro m
net/url.How ever, net/url mayimp ort net/http/httputil.
net/http
net/http/internal/chunked
net/http/httputil
net/url
10.7.6. Querying Packages
The go list to olrep ortsinfor mat ionabout avai lable packages. Inits simplestfor m, go list
testswhether a packageispresent inthe wor ksp ace andprintsits import pat h if so:
$golist github.com/go-sql-driver/mysql
github.com/go-sql-driver/mysql
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 10.7. THE GO TOOL 299
An argumentto go list maycontain the ‘‘...’’ wi ldc ard, whichmatch esany subst ringofa
packagesimp ort pat h. We can use ittoenumerateall the packages wit hin a Gowor ksp ace:
$golist ...
archive/tar
archive/zip
bufio
bytes
cmd/addr2line
cmd/api
...
many more
...
or wit hin a specificsubtree:
$golist gopl.io/ch3/...
gopl.io/ch3/basename1
gopl.io/ch3/basename2
gopl.io/ch3/comma
gopl.io/ch3/mandelbrot
gopl.io/ch3/netflag
gopl.io/ch3/printints
gopl.io/ch3/surface
or rel ate d to a par tic ulartopic:
$golist ...xml...
encoding/xml
gopl.io/ch7/xmlselect
The go list commandobt ainsthe completemet adat a foreachpackage, not justthe imp ort
path,and makes thisinfor mat ionavai lable tousers orother tools inavar ietyoffor mats. The
-json flag causes go list to print the ent ire record ofeachpackage in JSONfor mat:
$golist -json hash
{
"Dir": "/home/gopher/go/src/hash",
"ImportPath": "hash",
"Name": "hash",
"Doc": "Package hash provides interfaces for hash functions.",
"Target": "/home/gopher/go/pkg/darwin_amd64/hash.a",
"Goroot": true,
"Standard": true,
"Root": "/home/gopher/go",
"GoFiles": [
"hash.go"
],
"Imports": [
"io"
],
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
300 CHAPTER 10. PACKAGES AND THE GO TOOL
"Deps": [
"errors",
"io",
"runtime",
"sync",
"sync/atomic",
"unsafe"
]
}
The -f flag lets users customize the out put for mat usingthe templ ate langu ageofpackage
text/template (§4.6). Thiscommand printsthe transitive dep endencies ofthe strconv
package, sep arated byspaces:
$golist -f '{{join .Deps " "}}' strconv
errors math runtime unicode/utf8 unsafe
andthiscommand printsthe direc t importsofeachpackageinthe compress su btree ofthe
st and ard librar y:
$golist -f '{{.ImportPath}} -> {{join .Imports " "}}' compress/...
compress/bzip2 -> bufio io sort
compress/flate -> bufio fmt io math sort strconv
compress/gzip -> bufio compress/flate errors fmt hash hash/crc32 io time
compress/lzw -> bufio errors fmt io
compress/zlib -> bufio compress/flate errors fmt hash hash/adler32 io
The go list commandisusefulfor bot h on e-off int erac tivequer ies andfor bui ld andtest
automation scr ipts. Well use itagain in Sec tion 11.2.4. For moreinfor mat ion, includingthe
setofavai lable elds and their meaning, see the out put of go help list.
In thischapt er, weve explained all the imp ortantsub command s of the go to olexceptone.In
thenext chapt er, well see how the go test commandisusedfor testing Goprograms.
Exercis e 10.4: Cons truct a toolthatrep ortsthe set ofall packages in the wor ksp ace thattran-
sitive lydep endonthe packages specified bythe arguments. Hint: you will need torun
go list twice,oncefor the initial packages andoncefor all packages. You may wanttoparse
itsJSONout put usingthe encoding/json package(§4.5).
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
11
Te sting
Maur ice Wi l kes, the develop erofEDSAC, the rs t stored-program comp uter, had a startling
insig htwhi le climbing the stairsofhis lab orator y in 1949. In Memoir s of a ComputerPione er,
he recal le d, ‘‘Therealizat ioncameovermewit h full force thatagood par t of the remainder of
my lifewas going tobespent infindingerror s in myown programs.’’ Sure lyevery program-
merofastored-program comp utersince thencan symp athize wit h Wi l kes, thoug h perh aps
notwit houtsom e bemu sement at his naïvetéabout the difficult ies ofsof twarecon str uction.
Prog ramstoday arefar largerand more complex thaninWilkesstime, ofcours e, andagre at
de al of effor t hasbeenspent ontechniques tomakethiscomplexity manageable. Two
te chniques in par tic ularstand out for their effe ctiveness.The rs t is routine peerrevie w of
prog ramsbeforethe y aredeploye d.The secon d,the subjec t of thischapt er, istesting .
Test ing , by whichweimp licitlymean au tom ate d test ing , is the prac tice ofwriting small
prog ramsthatche ckthatthe codeunder test(the produc tion co de) behavesasexp ected for
cer tain inputs, whichare usu allyeit her caref ullychosentoexercis e cer tain featuresorran-
domize d to ens ure bro adcov erage.
The eldofsof twaretesting isenormou s.The taskoftesting occ upies allprogrammerssom e
of the timeand som e prog rammersall ofthe time. The lit eratureontesting includes
thou sands ofprint edbooks andmillions ofwords ofblog posts. Inevery mainst ream
prog ramminglangu age, there are dozensofsof twarepackages intended for testcon str uction,
some wit h agre atdealofthe ory,and the eldseems toatt rac t more thanafew pro phets wit h
cult-li kefol low ings. Itisalmostenoug h to convinceprogrammersthattowrite effec tive tests
they mustacquireawhole new setofski l ls.
Gosappro ach to test ing can seemrat her low-techincomparison. Itrelies onone command,
go test,and a set ofconvent ion s forwriting testfunctions that go test canrun. Thecom-
parat ive lylig htweig htmechanism iseffec tive for puretesting , anditextends natural lyto
benchmarks andsystematicexamples for documentation.
301
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
302 CHAPTER 11. TESTING
In prac tice,writing testcodeisnot muchdif ferentfro m wr iting the originalprogram its elf.
We w rite short functions thatfocus onone par t of the task. Wehavetobecaref ulofbound ary
condit ion s,thin k ab out dat a st ruc tures, andreasonabout whatresults a comp utation should
produce fro m suit ableinp uts. But thisisthe samepro cessaswriting ordinar y Go code; it
ne e dntrequirenew not ation s,convent ion s,and tools.
11.1. The go test To ol
The go test su bcommandisatestdriverfor Gopackages thatare organize d accordingtocer-
tain conv ent ion s.Inapackagedirec tor y,files whose names endwit h _test.go arenot par t of
thepackageordinar ily bui ltby go build butare a par t of it whenbui ltby go test.
Wi thin *_test.go files, three kinds offunctions are tre ate d sp eci ally : tests, benchmarks, and
examples. A te st func tion,whichisafunctionwhose namebeg inswit h Test,exercis essom e
prog ram log ic forcor rec t behavior ; go test callsthe testfunctionand rep ortsthe result,
whichiseit her PASS or FAIL.Abenchm ark fun cti on hasanamebeg inningwit h Benchmark
andmeasuresthe per for mance ofsom e op erat ion; go test reportsthe mean exec ution time
of the operat ion. And an ex amp lefun cti on,whose namestartswit h Example,provides
machine-che cke d do cumentation.Wewill cov ertests in detai l in Sec tion 11.2, benchmarksin
Section11.4, andexamples in Sec tion 11.6.
The go test to olscans the *_test.go files for these speci al func tions,generates a temporar y
main packagethatcal lsthemall inthe pro per way,bui ldsand runsit, rep ortsthe results, and
then cle ansup.
11.2. Test Functions
Each test file mustimp ort the testing package. Testfunctions havethe fol low ing sig nature:
func Test
Name
(t *testing.T) {
// ...
}
Test functionnames mustbeg in with Test;the opt ion alsuffix
Name
mu stbeg in with a capit al
letter:
func TestSin(t *testing.T) { /* ... */ }
func TestCos(t *testing.T) { /* ... */ }
func TestLog(t *testing.T) { /* ... */ }
The t parameter provides met hodsfor rep ortingtestfai lures andlog gingaddition al
infor mat ion. Letsdefine anexamplepackage gopl.io/ch11/word1,containinga singlefunc-
tion IsPalindrome that rep ortswhether a str ing reads the samefor wardand backward. (This
implementation tests every bytetwice if the str ing isapalindrom e;well com e back to that
shortly.)
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 11.2. TEST FUNCTIONS 303
gopl.io/ch11/word1
// Package word provides utilities for word games.
package word
// IsPalindrome reports whether s reads the same forward and backward.
// (Our first attempt.)
func IsPalindrome(s string) bool {
for i := range s {
if s[i] != s[len(s)-1-i] {
return false
}
}
return true
}
In the samedirec tor y,the file word_test.go cont ainstwo testfunctions named TestPalin-
drome and TestNonPalindrome.Eachche cks that IsPalindrome givesthe rig htanswerfor a
singleinp utand rep ortsfai lures using t.Error:
package word
import "testing"
func TestPalindrome(t *testing.T) {
if !IsPalindrome("detartrated") {
t.Error(`IsPalindrome("detartrated") = false`)
}
if !IsPalindrome("kayak") {
t.Error(`IsPalindrome("kayak") = false`)
}
}
func TestNonPalindrome(t *testing.T) {
if IsPalindrome("palindrome") {
t.Error(`IsPalindrome("palindrome") = true`)
}
}
A go test (or go build)command wit h no packagearguments operates onthe packagein
thecur rentdirec tor y.Wecan bui ld andrun the tests wit h thefol low ing command.
$cd$GOPATH/src/gopl.io/ch11/word1
$gotest
ok gopl.io/ch11/word1 0.008s
Sat isfied,weshipthe program, but nosoonerhavethe launch par tyguests depar ted thanthe
bugrep ortsstart toarrive . AFrenchusernamed NoelleEve Elleoncompl ainsthat IsPalin-
drome do esntrecog nize ‘‘été.’’ Anot her,fro m Cent ral America,isdis app ointe d that itrej e cts
‘‘Aman, a plan, a canal: Panama.’’ Thes e sp ecificand small bug rep ortsnatural lylendthem-
selves tonew testcas es.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
304 CHAPTER 11. TESTING
func TestFrenchPalindrome(t *testing.T) {
if !IsPalindrome("été") {
t.Error(`IsPalindrome("été") = false`)
}
}
func TestCanalPalindrome(t *testing.T) {
input := "A man, a plan, a canal: Panama"
if !IsPalindrome(input) {
t.Errorf(`IsPalindrome(%q) = false`,input)
}
}
To avoid writing the lon g input st ringtwice,weuse Errorf,whichprovides for matting like
Printf.
Wh enthe two new tests havebeenadde d,the go test commandfai lswit h infor mat ive error
mess ages.
$gotest
--- FAIL: TestFrenchPalindrome (0.00s)
word_test.go:28: IsPalindrome("été") = false
--- FAIL: TestCanalPalindrome (0.00s)
word_test.go:35: IsPalindrome("A man, a plan, a canal: Panama") = false
FAIL
FAIL gopl.io/ch11/word1 0.014s
Itsgood prac tice towrite the test rs t andobs erve thatittriggersthe samefai luredes crib edby
theusersbug rep ort.Onlythencan webeconfidentthatwhate ver xwecom e up wit h
addressesthe rig htpro blem.
As a bonus,running go test is usuallyquickerthanmanuallygoing through the steps
des crib edinthe bug rep ort,allow ing ustoiteratemorerapid ly. Ifthe testsuite containsmany
slow tests, wemay makeevenfasterprogress if were selec tive about whichoneswerun.
The -v flag printsthe nameand exe cut ion timeofeachtest in the package:
$gotest -v
=== RUN TestPalindrome
--- PASS: TestPalindrome (0.00s)
=== RUN TestNonPalindrome
--- PASS: TestNonPalindrome (0.00s)
=== RUN TestFrenchPalindrome
--- FAIL: TestFrenchPalindrome (0.00s)
word_test.go:28: IsPalindrome("été") = false
=== RUN TestCanalPalindrome
--- FAIL: TestCanalPalindrome (0.00s)
word_test.go:35: IsPalindrome("A man, a plan, a canal: Panama") = false
FAIL
exit status 1
FAIL gopl.io/ch11/word1 0.017s
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 11.2. TEST FUNCTIONS 305
andthe -run flag , whos e argumentisaregu lar expression, causes go test to run onlythose
testswhose functionnamematch esthe pattern:
$gotest -v -run="French|Canal"
=== RUN TestFrenchPalindrome
--- FAIL: TestFrenchPalindrome (0.00s)
word_test.go:28: IsPalindrome("été") = false
=== RUN TestCanalPalindrome
--- FAIL: TestCanalPalindrome (0.00s)
word_test.go:35: IsPalindrome("A man, a plan, a canal: Panama") = false
FAIL
exit status 1
FAIL gopl.io/ch11/word1 0.014s
Of course,onceweve gott enthe selec ted tests topass, weshouldinv oke go test with no flags
to run the ent ire testsuite one lasttimebeforewecommitthe change .
No w ourtaskistofixthe bugs. A quickinv est igat ionreveals the cause ofthe rs t bugtobe
IsPalindromesuse ofbytesequences, not runesequences, sothatnon-ASCII charac terssuch
as the é in "été" conf use it. Thesecon d bugarisesfro m notignor ing spaces, punctuation,
andlettercas e.
Chastene d,werewrite the functionmorecaref ully:
gopl.io/ch11/word2
// Package word provides utilities for word games.
package word
import "unicode"
// IsPalindrome reports whether s reads the same forward and backward.
// Letter case is ignored, as are non-letters.
func IsPalindrome(s string) bool {
var letters []rune
for _, r := range s {
if unicode.IsLetter(r) {
letters = append(letters, unicode.ToLower(r))
}
}
for i := range letters {
if letters[i] != letters[len(letters)-1-i] {
return false
}
}
return true
}
We als o wr ite a morecomprehensiveset oftestcas es that combines allthe pre vious onesand a
numb erofnew onesint o atable.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
306 CHAPTER 11. TESTING
func TestIsPalindrome(t *testing.T) {
var tests = []struct {
input string
want bool
}{
{"", true},
{"a", true},
{"aa", true},
{"ab", false},
{"kayak", true},
{"detartrated", true},
{"A man, a plan, a canal: Panama", true},
{"Evil I did dwell; lewd did I live.", true},
{"Able was I ere I saw Elba", true},
{"été", true},
{"Et se resservir, ivresse reste.", true},
{"palindrome", false}, // non-palindrome
{"desserts", false}, // semi-palindrome
}
for _, test := range tests {
if got := IsPalindrome(test.input); got != test.want {
t.Errorf("IsPalindrome(%q) = %v", test.input, got)
}
}
}
Ournew tests pass:
$gotest gopl.io/ch11/word2
ok gopl.io/ch11/word2 0.015s
Thissty leof tabl e-driven test ing isver y common inGo. Itisstraig htfor wardtoadd new table
entr ies as needed,and since the ass ertionlog ic is not dup lic ated,wecan invest moreeffor t in
producingagood error message .
Theout put of a fai lingtestdoes not include the ent ire stack trace at the moment ofthe cal l to
t.Errorf.Nor does t.Errorf caus e apanic orstopthe exe cut ion of the test, unli keass ertion
fai lures in manytestframewor ksfor other langu ages. Tests areindep endentofeachother.If
an early ent ryinthe table causesthe testtofai l,later table entr ies will still beche cke d,and
thus wemay lear n ab out multiplefai lures dur ingasinglerun.
Wh enwereallymuststopatestfunction, perhaps because some initializat ioncodefai le d or to
preventafai lurealready rep orted fro m causingaconfusingcas cade ofothers, weuse t.Fatal
or t.Fatalf.These mustbecal le d from the samegoroutine asthe Test func tion, not fro m
anot her one cre ate d during the test.
Test fai luremessagesare usu allyofthe for m "f(x) = y, want z",where f(x) explainsthe
attemp ted operat ionand its input, y is the actualresult, and z theexp ected result. Where con-
venient, as in our palindrom e example, actualGosyntaxisusedfor the f(x) part.Displ aying
x is par tic ularlyimp ortant in a table-dr iventest, since a given assertionisexe cut edmany
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 11.2. TEST FUNCTIONS 307
timeswit h dif ferentvalues. Avoid boi ler plate and redundant infor mat ion. Whentesting a
boole an func tionsuchas IsPalindrome,omitthe want z part since itaddsnoinfor mat ion. If
x, y,or z is lengt hy, print a con cis e summar y of the relevantpar ts instead. The aut hor of a test
shouldstr ive tohelpthe programmer who mustdiagnos e atestfai lure.
Exercis e 11.1: Wr ite testsfor the charcount prog ram in Sec tion 4.3.
Exercis e 11.2: Wr ite a set oftests for IntSet (§6.5) thatche cks thatits beh avior after each
op erat ionisequivalenttoaset bas edonbui lt-in maps. Saveyourimp lementation for
benchmarking inExercis e 11.7.
11.2.1. Randomized Testing
Table-dr iventests areconvenientfor che cking thatafunctionwor ksoninp uts caref ully
sele cte d to exercis e interest ing cas es in the log ic. Anot her approach, ra ndomizedtesting,
exploresabro ader range ofinp uts bycon str uctinginp uts at random.
Ho w do weknowwhatout put toexp ect fro m ourfunction, given a randominp ut? Thereare
twostrateg ies. The rs t is towrite analt ernat ive imp lementation of the functionthatusesa
lessefficientbut simpler andcle arer algor it hm, andche ckthatbot h implementation s give the
same result. Thesecon d is tocre ate inp utvalues accordingtoapattern sothatweknowwhat
output toexp ect.
Theexamplebelow usesthe secon d approach:the randomPalindrome func tiongenerates
word s that areknown tobepalindrom esbycon str uction.
import "math/rand"
// randomPalindrome returns a palindrome whose length and contents
// are derived from the pseudo-random number generator rng.
func randomPalindrome(rng *rand.Rand) string {
n:=rng.Intn(25) // random length up to 24
runes := make([]rune, n)
for i := 0; i < (n+1)/2; i++ {
r:=rune(rng.Intn(0x1000)) // random rune up to '\u0999'
runes[i] = r
runes[n-1-i] = r
}
return string(runes)
}
func TestRandomPalindromes(t *testing.T) {
// Initialize a pseudo-random number generator.
seed := time.Now().UTC().UnixNano()
t.Logf("Random seed: %d", seed)
rng := rand.New(rand.NewSource(seed))
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
308 CHAPTER 11. TESTING
for i := 0; i < 1000; i++ {
p:=randomPalindrome(rng)
if !IsPalindrome(p) {
t.Errorf("IsPalindrome(%q) = false", p)
}
}
}
Sincerandomize d testsare non deter minist ic, itiscriticalthatthe log ofthe fai lingtestrecord
sufficientinfor mat iontorepro duce the fai lure. Inour example, the inp ut p to IsPalindrome
te lls us all weneed toknow, but for functions thatacceptmorecomplex inputs, itmay besim-
pler tolog the seed ofthe pseudo-randomnumbergenerator (as wedoabove)thantodump
theent ire inp utdat a st ruc ture. Armed wit h that seed value,wecan easi lymodif y thetestto
repl aythe fai luredeter minist ically.
By usingthe cur renttimeasasourceofrandomness, the testwill explore nov elinp uts each
time itisrun, overthe ent ire cours e of itslifet ime. Thisisesp eci ally valuableifyourpro jec t
us es an aut omated systemtorun allits tests per iodic ally.
Exercis e 11.3: TestRandomPalindromes on lytests palindrom es. Write a randomize d test that
generates andver ifies non-palindrom es.
Exercis e 11.4: Mo dif y randomPalindrome to exercis e IsPalindromeshandlingofpunc-
tu ation and spaces.
11.2.2. Testing a Command
The go test to olisusefulfor testing librar y packages, but wit h alit tle effort wecan use itto
test command s as wel l.Apackagenamed main ordinar ily pro duces an exec utableprogram,
butitcan beimp orted asalibrar y to o.
Letswrite a test for the echo prog ram of Sec tion 2.3.2. Weve split the program into two func-
tion s: echo do es therealwor k,whi le main pars esand reads the flag values andrep ortsany
er ror s returned by echo.
gopl.io/ch11/echo
// Echo prints its command-line arguments.
package main
import (
"flag"
"fmt"
"io"
"os"
"strings"
)
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 11.2. TEST FUNCTIONS 309
var (
n=flag.Bool("n", false, "omit trailing newline")
s=flag.String("s", " ", "separator")
)
var out io.Writer = os.Stdout // modified during testing
func main() {
flag.Parse()
if err := echo(!*n, *s, flag.Args()); err != nil {
fmt.Fprintf(os.Stderr, "echo: %v\n", err)
os.Exit(1)
}
}
func echo(newline bool, sep string, args []string) error {
fmt.Fprint(out, strings.Join(args, sep))
if newline {
fmt.Fprintln(out)
}
return nil
}
Fr omthe test, wewill cal l echo with a var ietyofarguments and flag settingsand che ckthatit
pr intsthe cor rec t output ineachcas e,soweve adde d parametersto echo to reduce its depen-
dence onglobalvar iables. That said,weve als o introduce d anot her globalvar iable, out,the
io.Writer to whichthe resultwill bewritt en. Byhav ing echo wr ite through thisvar iable,not
direc tly to os.Stdout,the tests can subst itute a dif ferent Writer implementation thatrecords
what was writt enfor later insp ection. Heresthe test, in file echo_test.go:
package main
import (
"bytes"
"fmt"
"testing"
)
func TestEcho(t *testing.T) {
var tests = []struct {
newline bool
sep string
args []string
want string
}{
{true, "", []string{}, "\n"},
{false, "", []string{}, ""},
{true, "\t", []string{"one", "two", "three"}, "one\ttwo\tthree\n"},
{true, ",", []string{"a", "b", "c"}, "a,b,c\n"},
{false, ":", []string{"1", "2", "3"}, "1:2:3"},
}
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
310 CHAPTER 11. TESTING
for _, test := range tests {
descr := fmt.Sprintf("echo(%v, %q, %q)",
test.newline, test.sep, test.args)
out = new(bytes.Buffer) // captured output
if err := echo(test.newline, test.sep, test.args); err != nil {
t.Errorf("%s failed: %v", descr, err)
continue
}
got := out.(*bytes.Buffer).String()
if got != test.want {
t.Errorf("%s = %q, want %q", descr, got, test.want)
}
}
}
No tice thatthe testcodeisinthe samepackageasthe pro duc tioncode. Alt hough the package
name is main anditdefinesamain func tion, dur ingtesting thispackageactsasalibrar y that
exp oses the function TestEcho to the testdriver; its main func tionisignored.
By organizingthe testasatable,wecan easi lyadd new testcas es. Letssee whathappens when
thetestfai ls, byaddingthislinetothe table:
{true, ",", []string{"a", "b", "c"}, "a b c\n"}, // NOTE: wrong expectation!
go test pr ints
$gotest gopl.io/ch11/echo
--- FAIL: TestEcho (0.00s)
echo_test.go:31: echo(true, ",", ["a" "b" "c"]) = "a,b,c", want "a b c\n"
FAIL
FAIL gopl.io/ch11/echo 0.006s
Theerror message des crib esthe att emp ted operat ion(usingGo-li kesyntax), the actualbeh av-
ior, and the exp ected beh avior,inthatorder.Wit h an infor mat ive error message suchasthis,
youmay haveaprett y go o d ide a ab out the rootcause beforeyouve evenlocated the source
co de of the test.
Itsimp ortantthatcodebeingteste d notcal l log.Fatal or os.Exit,since these will stopthe
processinits tracks; cal lingthese functions shouldberegarde d as the exc lusiverig htof main.
If som ethingtot allyunexp ected happens and a functionpanics, the testdriverwill recov er,
though the testwill ofcours e be con sidered a fai lure. Exp ected error s such asthose resulting
from bad userinp ut, missing files, orimp rop erconfigurat ionshouldberep orted byretur ning
anon-ni l error value.For tunately(though unfor tunateasanillustrat ion), our echo example
is sosimplethatitwill never retur n anon-ni l er ror.
11.2.3. White-Box Testing
Oneway ofcategor izingtests isbythe level ofknowledge the y re quireofthe int ernal wor kings
of the packageunder test. A bl ack-box test assumesnot hingabout the packageother than
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 11.2. TEST FUNCTIONS 311
what isexp osed byits APIand specified byits document ation;the packagesint ernalsare
op aque. Incontrast, a whit e-box test has privi leged accesstothe int ernal functions and dat a
st ruc tures ofthe packageand can makeobs ervat ions and changesthatanordinar y clientcan-
not. For example, a white-b oxtestcan che ckthatthe invar iants ofthe packagesdat a typesare
maintained after every operat ion. (Thename whit e box is tradition al, but cl ear box wouldbe
more acc urate.)
Thetwo appro ach esare complementary.Black-b oxtests areusu allymorerobust, needing
fe wer updates as the sof twareevo l ves. They als o helpthe testaut hor emp athize wit h theclient
of the packageand can reveal flaws in the API desig n. In contrast, white-b oxtests can provide
more det aile d coverageofthe trick ier par ts of the imp lementation.
Weve already seenexamples ofbot h kind s. TestIsPalindrome callsonlythe exp orted func-
tion IsPalindrome andisthu s ablack-b oxtest. TestEcho callsthe echo func tionand
up dates the globalvar iable out,bot h of whichare unexp orted,mak ingitawhite-b oxtest.
Whiledeveloping TestEcho,wemodified the echo func tiontouse the package-le vel var iable
out when writing its out put,sothatthe testcou ldreplace the stand ard out put wit h an alter-
native imp lementation thatrecords the dat a forlater insp ection. Usingthe sametechnique,
we can replace other par ts of the pro duc tioncodewit h easy-to-test ‘‘fake’’ implementation s.
Theadvantage offakeimp lementation s is thatthe y canbesimpler toconfigure, more
predic table,morereliable, and easier toobs erve . They can also avoid undesirable side effe cts
such asupdat ingapro duc tiondat abas e or charg ing a cre dit card.
Thecodebelow shows the quota-che cking log ic in a web ser vice thatprovides net wor ked
storagetousers.Whenusers exceed 90% oftheir quota,the systemsends themawar ning
emai l.
gopl.io/ch11/storage1
package storage
import (
"fmt"
"log"
"net/smtp"
)
func bytesInUse(username string) int64 { return 0 /* ... */ }
// Email sender configuration.
// NOTE: never put passwords in source code!
const sender = "notifications@example.com"
const password = "correcthorsebatterystaple"
const hostname = "smtp.example.com"
const template = `Warning: you are using %d bytes of storage,
%d%% of your quota.`
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
312 CHAPTER 11. TESTING
func CheckQuota(username string) {
used := bytesInUse(username)
const quota = 1000000000 // 1GB
percent := 100 * used / quota
if percent < 90 {
return // OK
}
msg := fmt.Sprintf(template, used, percent)
auth := smtp.PlainAuth("", sender, password, hostname)
err := smtp.SendMail(hostname+":587", auth, sender,
[]string{username}, []byte(msg))
if err != nil {
log.Printf("smtp.SendMail(%s) failed: %s", username, err)
}
}
Wedliketotestit, but wedontwantthe testtosendout realemail.Sowemov e theemail
log ic into its own functionand store thatfunctioninanunexp orted package-le vel var iable,
notifyUser.
gopl.io/ch11/storage2
var notifyUser = func(username, msg string) {
auth := smtp.PlainAuth("", sender, password, hostname)
err := smtp.SendMail(hostname+":587", auth, sender,
[]string{username}, []byte(msg))
if err != nil {
log.Printf("smtp.SendEmail(%s) failed: %s", username, err)
}
}
func CheckQuota(username string) {
used := bytesInUse(username)
const quota = 1000000000 // 1GB
percent := 100 * used / quota
if percent < 90 {
return // OK
}
msg := fmt.Sprintf(template, used, percent)
notifyUser(username, msg)
}
We can now write a test thatsubst itutesasimplefakenot ication mechanism insteadofsend-
ingrealemail.Thisone records the notied userand the contentsofthe mess age .
package storage
import (
"strings"
"testing"
)
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 11.2. TEST FUNCTIONS 313
func TestCheckQuotaNotifiesUser(t *testing.T) {
var notifiedUser, notifiedMsg string
notifyUser = func(user, msg string) {
notifiedUser, notifiedMsg = user, msg
}
// ...simulate a 980MB-used condition...
const user = "joe@example.org"
CheckQuota(user)
if notifiedUser == "" && notifiedMsg == "" {
t.Fatalf("notifyUser not called")
}
if notifiedUser != user {
t.Errorf("wrong user (%s) notified, want %s",
notifiedUser, user)
}
const wantSubstring = "98% of your quota"
if !strings.Contains(notifiedMsg, wantSubstring) {
t.Errorf("unexpected notification message <<%s>>, "+
"want substring %q", notifiedMsg, wantSubstring)
}
}
Theresone pro blem: afterthistestfunctionhas retur ned, CheckQuota no lon g erwor ksasit
shouldbecause itsstill usingthe testsfakeimp lementation of notifyUsers.(Thereisalways
ariskofthiskindwhenupdat ingglobalvar iables.) Wemustmodif y thetesttorestore the
previous value sothatsubsequenttests obs erve noeffec t,and wemustdothisonall exe cut ion
paths, includingtestfai lures andpanics. Thisnatural lysug gests defer.
func TestCheckQuotaNotifiesUser(t *testing.T) {
// Save and restore original notifyUser.
saved := notifyUser
defer func() { notifyUser = saved }()
// Install the test'sfake notifyUser.
var notifiedUser, notifiedMsg string
notifyUser = func(user, msg string) {
notifiedUser, notifiedMsg = user, msg
}
// ...rest of test...
}
Thispattern can beusedtotemporar ily saveand restore all kinds ofglobalvar iables, including
command-line flags, debuggingopt ion s,and per for mance parameters; toins tal l andremov e
ho oks thatcause the pro duc tioncodetocal l some testcodewhensom ethingint erest ing hap-
pens;and tocoaxthe pro duc tioncodeint o rarebut imp ortantstates, such as timeouts, erro rs,
andevenspecificint erleavingsofcon cur rentactiv ities.
Usingglobalvar iables in thisway issafeonlybecause go test do es notnor mal lyrun multiple
testscon cur rently.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
314 CHAPTER 11. TESTING
11.2.4. External Test Packages
Consider the packages net/url,whichprovides a URL parser, and net/http,whichprovides
aweb ser ver andHTTPclientlibrar y.Aswemig htexp ect,the hig her-le vel net/http dep ends
on the low er-le vel net/url.How ever, one ofthe tests in net/url is anexampledemon strat-
ingthe int erac tionbet weenURLs andthe HTTPclientlibrar y.Inother words,atestofthe
lower-le vel packageimp ortsthe hig her-le vel package.
Figure 11.1. Atestof net/url dep ends on net/http.
Decl aring thistestfunctioninthe net/url packagewou ldcre ate a cyc leinthe package
import graph, as depic ted bythe upwards arrow inFigure11.1, but asweexplained in
Section10.1, the Gospecification forbids imp ort cyc les.
We res olvethe pro blem bydeclaring the testfunctioninan exter nal testpackage,thatis, in a
file in the net/url direc tor y whos e packagedeclarat ionreads package url_test.The ext ra
suffix _test is a sig nal to go test that itshouldbui ld an addition alpackagecontainingjust
thes e files andrun its tests. Itmay behelpf ultothin k of thisexter nal testpackageasifithad
theimp ort pat h net/url_test,but it cannot beimp orted under thisorany other name.
Becaus e exter nal tests live inasep aratepackage, the y mayimp ort helperpackages thatals o
dep endonthe packagebeingteste d;anin-p ackagetestcannot dothis. Inter msofthe desig n
layers,the exter nal testpackageislog ical lyhig her upthanbot h of the packages itdep ends
up on, as shown inFigure11.2.
Figure 11.2. External testpackages bre akdep endency cycles.
By avoidingimp ort cyc les, exter nal testpackages allowtests, especi ally inte grati ontests (w hich
test the int erac tionofseveral comp onents), toimp ort other packages fre ely,exac tly asan
applic ationwou ld.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 11.2. TEST FUNCTIONS 315
We c an use the go list to oltosummar ize whichGosourcefiles in a packagedirec tor y are
produc tioncode, in-p ackagetests, andexter nal tests. Well use the fmt packageasanexample.
GoFiles is the listoffiles thatcontain the pro duc tioncode; these are the files that go build
wi l l include in yourapp lic ation:
$golist -f={{.GoFiles}} fmt
[doc.go format.go print.go scan.go]
TestGoFiles is the listoffiles thatals o belong tothe fmt package, but these files, whose
namesall end in _test.go,are include d on lywhenbui ldingtests:
$golist -f={{.TestGoFiles}} fmt
[export_test.go]
Thepackagestests wou ldusu allyreside in these files, thoug h unusually fmt hasnon e;well
explain the pur pos e of export_test.go in a mom ent.
XTestGoFiles is the listoffiles thatcon stitute the exter nal testpackage, fmt_test,sothese
files mustimp ort the fmt packageinorder touse it. Again, the y areinclude d on lydur ingtest-
ing:
$golist -f={{.XTestGoFiles}} fmt
[fmt_test.go scan_test.go stringer_test.go]
Sometimesanexter nal testpackagemay need privi leged accesstothe int ernalsofthe package
under test, if for exampleawhite-b oxtestmustliveinasep aratepackagetoavoid an import
cycle. Insuchcas es, we use a trick:weadd decl arat ions toanin-p ackage _test.go file to
exp osethe necessary int ernalstothe exter nal test. This file thu s of fersthe testa‘‘back door’’
to the package. Ifthe sourcefile existsonlyfor thispur pos e andcontainsnotests its elf,itis
of ten cal le d export_test.go.
Fo r example, the imp lementation of the fmt packageneedsthe functionality of unicode.Is-
Space as par t of fmt.Scanf.Toavoid creating anundesirable dependency, fmt do es not
import the unicode packageand its large tables ofdat a; instead, itcontainsasimpler imple-
ment ation,whichitcal ls isSpace.
To ens ure thatthe beh avior s of fmt.isSpace and unicode.IsSpace do not drift apart, fmt
pr udentlycontainsatest. Itisanexter nal test, andthu s it cannot access isSpace direc tly,so
fmt op ens a backdoortoitbydeclaring anexp orted var iable thatholds the int ernal isSpace
func tion. Thisisthe ent irety ofthe fmt packages export_test.go file.
package fmt
var IsSpace = isSpace
Thistest file definesnotests; itjustdeclaresthe exp orted symbol fmt.IsSpace foruse bythe
exter nal test. Thistrick can also beusedwhene ver an exter nal testneedstouse some ofthe
te chniques ofwhite-b oxtesting .
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
316 CHAPTER 11. TESTING
11.2.5. Writing EffectiveTests
Many newcomers to Goare sur prisedbythe minimalism ofGostesting framewor k.Other
languages framewor ksprovide mechanismsfor identifyingtestfunctions (of ten usingreflec-
tion or met adat a), ho oks for per for ming ‘‘setup’’ and ‘‘te ardow n’’ op erat ions beforeand after
thetests run, andlibrar ies of utilit y func tions for ass ertingcommonpre dic ates, comp aring
values, for matting error messages, andabortingafai le d test (of ten usingexception s).
Although these mechanismscan maketests ver y concis e,the resulting tests often seemlike
they are writt eninaforeig n language . Furthermore, alt hough the y mayrep ort PASS or FAIL
correc tly,their manner may beunf riendlytothe unfor tunatemaintainer,wit h cr ypticfai lure
mess ageslike "assert: 0 == 1" or pageafter pageofstack traces.
Gosatt itude totesting stand s in stark contrast. Itexp ectstestaut hor s to domostofthiswor k
themselves, definingfunctions toavoid rep etit ion,justasthe y wouldfor ordinar y prog rams.
Thepro cessoftesting isnot one ofrot e form lling; a testhas a userint erface too,alb eit one
whos e on lyusers are als o itsmaintainers. A go o d test doesnot explo de on fai lurebut prints a
clearand succinct des crip tionofthe sympt omofthe pro blem, andperhaps other relevant
fac ts ab out the context. Ide ally,the maintainer shouldnot need toreadthe sourcecodeto
de ciph eratestfai lure. A go o d test shouldnot giveupafter one fai lurebut shouldtry torep ort
several erro rs in a singlerun, since the pattern offai lures may its elf berevealing.
Theass ertionfunctionbelow comparestwo values, cons tructsagener ic er ror message , and
stopsthe program. Itseasy touse and itscor rec t,but whenitfai ls, the error message isalmost
us eless. Itdoesnot solvethe hardpro blem ofprovidingagood userint erface.
import (
"fmt"
"strings"
"testing"
)
// A poor assertion function.
func assertEqual(x, y int) {
if x != y {
panic(fmt.Sprintf("%d != %d", x, y))
}
}
func TestSplit(t *testing.T) {
words := strings.Split("a:b:c", ":")
assertEqual(len(words), 3)
// ...
}
In thissense,ass ertionfunctions suf fer fro m premat ure abstrac tion:bytre ating the fai lureof
this par tic ulartestasameredif ference oftwo int egers,wefor feitthe opp ortunity toprovide
me aningf ul cont ext. Wecan provide a bettermessage bystartingfro m thecon crete det ails,as
in the examplebelow.Onlyoncerep etit ive patternsemerge inagiven testsuite isittimeto
introduce abstrac tions.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 11.2. TEST FUNCTIONS 317
func TestSplit(t *testing.T) {
s, sep := "a:b:c", ":"
words := strings.Split(s, sep)
if got, want := len(words), 3; got != want {
t.Errorf("Split(%q, %q) returned %d words, want %d",
s, sep, got, want)
}
// ...
}
No w thetestrep ortsthe functionthatwas cal le d,its inputs, andthe sig nificanceofthe result;
it explicitlyidentifies the actualvalue andthe exp ectat ion; anditcontinues toexe cut e even if
this ass ertionshouldfai l.Onceweve writt enatestlikethis, the natural next stepisoften not
to define a functiontoreplace the ent ire if st atement,but toexe cut e thetestinaloopin
which s, sep,and want var y,likethe table-dr iventestof IsPalindrome.
Thepre vious exampledidntneed any utilit y func tions,but of cours e that shouldntstopus
from int roducingfunctions whenthe y helpmakethe codesimpler.(Well look at one such
ut i lit y func tion, reflect.DeepEqual,inSec tion 13.3.) Thekey toagood testistostart by
implementing the con crete beh avior thatyou wantand onlythenuse functions tosimplif y the
co de andeliminaterep etit ion.Bestresults arerarelyobt ained bystartingwit h alibrar y of
abstrac t, generictesting functions.
Exercis e 11.5: Extend TestSplit to use a table ofinp uts andexp ected out puts.
11.2.6. Avoiding Brittle Tests
An app lic ationthatoften fails whenitencount ers new but valid inputs iscal le d bug gy;atest
that spuriou sly fai lswhenasound change was made tothe program iscal le d br ittle.Justasa
bugg y prog ram frust rates its users,abritt letestexasperates its maintainers. Themostbritt le
tests, whichfai l foralmostany change tothe pro duc tioncode, good orbad,are som etimes
called ch angedetec tor or statu s qu o tests, andthe timespent dealingwit h them can quickly
depleteany benefitthe y on ceseeme d to provide.
Wh enafunctionunder testpro duces a comp lex out put suchasalon g st ring, anelaboratedat a
st ruc ture, orafile,itstempt ing toche ckthatthe out put isexac tly equ altosom e ‘‘go lden’’ value
that was expec ted whenthe testwas writt en. But asthe program evo l ves, par ts of the out put
wi l l li kelychange , prob ably ingood ways, but change non etheless. And itsnot justthe out put;
func tions wit h comp lex inputs often bre akbecause the inp utused in a testisnolon g ervalid.
Theeasiestway toavoid britt letests istoche ckonlythe pro per ties you careabout.Testyour
prog ramssimpler andmorestableint erfaces in preferencetoits internal functions.Beselec-
tive inyourass ertions.Dontche ckfor exac t st ringmatch es, for example, but lookfor relevant
su bst rings thatwill remain unch ange d as the program evo l ves. Itsoften wor thwriting a
su bst ant ial functiontodistill a complex out put dow n to its essencesothatass ertions will be
re liable. Eventhoug h that may seemlikealot ofup-f ronteffor t,itcan pay for itself quicklyin
time thatwou ldother wis e be spent fixingspuriou sly fai lingtests.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
318 CHAPTER 11. TESTING
11.3. Coverage
By its nature, test ing isnever comp lete. Asthe influential comp uterscientist EdsgerDijkstra
putit, ‘‘Te st ing shows the presence, not the abs ence ofbugs.’’ No quant ity oftests can ever
prov e apackagefre e of bugs. Atbest, the y increase our confidence thatthe packagewor ks
we ll in a wide range ofimp ortantscenar ios.
Thedeg ree towhichatestsuite exercis esthe packageunder testiscal le d thetests covera ge.
Coveragecantbequant ied direc tlythedynamics ofall but the most trivialprogramsare
beyond pre cis e me asurementbutthere are heur ist ics thatcan helpusdirec t ourtesting
ef for tstowhere the y aremorelikelytobeuseful.
St atement cov era ge is the simplestand most widely usedofthese heur ist ics. Thestatement
coverageofatestsuite isthe frac tionofsourcestatementsthatare exe cut edatleast oncedur-
ingthe test. Inthissec tion,well use Gos cover to ol, whichisint egrated int o go test,to
me asure statement cov erageand helpidentify obv iou s gaps in the tests.
Thecodebelow isatable-dr iventestfor the expressionevaluatorwebui ltback in Chapt er7:
gopl.io/ch7/eval
func TestCoverage(t *testing.T) {
var tests = []struct {
input string
env Env
want string // expected error from Parse/Check or result from Eval
}{
{"x % 2", nil, "unexpected '%'"},
{"!true", nil, "unexpected '!'"},
{"log(10)", nil, `unknown function "log"`},
{"sqrt(1, 2)", nil, "call to sqrt has 2 args, want 1"},
{"sqrt(A / pi)", Env{"A": 87616, "pi": math.Pi}, "167"},
{"pow(x, 3) + pow(y, 3)", Env{"x": 9, "y": 10}, "1729"},
{"5 / 9 * (F - 32)", Env{"F": -40}, "-40"},
}
for _, test := range tests {
expr, err := Parse(test.input)
if err == nil {
err = expr.Check(map[Var]bool{})
}
if err != nil {
if err.Error() != test.want {
t.Errorf("%s: got %q, want %q", test.input, err, test.want)
}
continue
}
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 11.3. COVERAGE 319
got := fmt.Sprintf("%.6g", expr.Eval(test.env))
if got != test.want {
t.Errorf("%s: %v => %s, want %s",
test.input, test.env, got, test.want)
}
}
}
First,letsche ckthatthe testpasses:
$gotest -v -run=Coverage gopl.io/ch7/eval
=== RUN TestCoverage
--- PASS: TestCoverage (0.00s)
PASS
ok gopl.io/ch7/eval 0.011s
Thiscommand displ ays the usage message ofthe cov eragetool:
$gotool cover
Usage of 'go tool cover':
Given a coverage profile produced by 'go test':
go test -coverprofile=c.out
Open a web browser displaying annotated source code:
go tool cover -html=c.out
...
The go tool commandrunsone ofthe exe cut ables fro m theGotoolchain. Thes e prog rams
live inthe direc tor y $GOROOT/pkg/tool/${GOOS}_${GOARCH}.Thanksto go build,werarely
ne e d to inv oke themdirec tly.
No w we run the testwit h the -coverprofile flag:
$gotest -run=Coverage -coverprofile=c.out gopl.io/ch7/eval
ok gopl.io/ch7/eval 0.032s coverage: 68.5% of statements
This flag enables the col lec tion of cov eragedat a by inst rum ent ing thepro duc tioncode. That
is,itmodifies a copy ofthe sourcecodesothatbeforeeachblo ckofstatementsisexe cut ed, a
boole an var iable isset, wit h on e var iable per blo ck. Justbeforethe modified program exits, it
wr itesthe value ofeachvar iable tothe specified log file c.out andprintsasummar y of the
frac tionofstatementsthatwereexe cut ed. (If all you need isthe summar y,use
go test -cover.)
If go test is run wit h the -covermode=count flag , theins trument ation for eachblo ckincre-
mentsacount erins teadofsetting a boole an. Theresulting log ofexe cut ion countsofeach
blockenables quant itative comparisons bet ween ‘‘hott er’’ blocks, whichare morefre quently
exec ute d, and ‘‘colder’’ on es.
Having gat hered the dat a, we run the cover to ol, whichpro cessesthe log , generates an HTML
report,and opens it in a new brows er window (Figure11.3).
$gotool cover -html=c.out
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
320 CHAPTER 11. TESTING
Figure 11.3. Acov eragerep ort.
Each statement iscolored greenifitwas covere d or red ifitwas not cov ere d.For clarity,weve
sh ade d thebackg round ofthe red text. Wecan see immediate lythatnon e of our inputs exer-
cisedthe unary operator Eval method.Ifweadd thisnew testcas e to the table andre-r un the
previous two command s,the unary expressioncodebecom esgre en:
{"-x * -x", eval.Env{"x": 2}, "4"}
Thetwo panic st atementsremain red,how ever. Thisshouldnot besur prising, because these
st atementsare sup pos edtobeunreach able.
Ac hie ving100% statement cov eragesound s li keanoble goal, but it isnot usu allyfeasiblein
prac tice,nor isitlikelytobeagood use ofeffor t.Justbecause a statement isexe cut eddoes
notmeanitisbug-f ree;statementscontainingcomplex expressions mustbeexe cut edmany
timeswit h dif ferentinp uts tocov erthe int erest ing cas es. Some statements, like the panic
st atementsabove , cannever bereach ed. Others, suchasthose thathandleesotericerror s,are
hard toexercis e butrarelyreach edinprac tice.Testing isfundament allyaprag mat ic ende avor,
atrade-off bet weenthe costofwriting tests andthe costoffai lures thatcou ldhavebeen
prevente d by tests. Cov eragetools can helpidentify the weakest spots, but devisinggood test
casesdemands the samerigorou s thin king as programming in general.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 11.4. BENCHMARK FUNCTIONS 321
11.4. Benchmark Functions
Benchmarking isthe prac tice ofmeasuring the per for mance ofaprogram onaxe d
work load. InGo, a benchmark functionlooks like a testfunction, but wit h the Benchmark
prefixand a *testing.B parameter thatprovides mostofthe samemet hodsasa*testing.T,
plus a few ext ra re lated toper for mance measurement.Itals o exp oses an integerfield N,which
sp ecifies the numberoftimes toper for m theoperat ionbeingmeasure d.
Heresabenchmark for IsPalindrome that cal lsit N times in a loop.
import "testing"
func BenchmarkIsPalindrome(b *testing.B) {
for i := 0; i < b.N; i++ {
IsPalindrome("A man, a plan, a canal: Panama")
}
}
We run itwit h thecommand below.Unliketests, bydefau ltnobenchmarksare run. The
argumenttothe -bench flag selec tswhichbenchmarkstorun. Itisaregu lar expression
matchingthe names of Benchmark func tions,wit h adefau ltvalue thatmatch esnon e of them.
The ‘‘.’’ patt ern causesittomatch all benchmarksinthe word package, but since theresonly
on e, -bench=IsPalindrome wouldhavebeenequivalent.
$cd$GOPATH/src/gopl.io/ch11/word2
$gotest -bench=.
PASS
BenchmarkIsPalindrome-8 1000000 1035 ns/op
ok gopl.io/ch11/word2 2.179s
Thebenchmark namesnumer ic suffix, 8 here , indic ates the value of GOMAXPROCS,whichis
importantfor con cur rentbenchmarks.
Therep ort tel lsusthateachcal l to IsPalindrome to okabout 1.035 micros econd s,averaged
ov er1,000,000 runs. Since the benchmark runner initial lyhas noide a howlon g theoperat ion
takes, itmakes som e initial measurementsusingsmall values of N andthenext rapol atestoa
value large enoug h forastabletimingmeasurement tobemade.
Thereasonthe loopisimp lemente d by the benchmark function, andnot bythe cal lingcodein
thetestdriver, issothatthe benchmark functionhas the opp ortunity toexe cut e anynecessary
on e-t imesetup codeoutside the loopwit houtthisaddingtothe measured time ofeachitera-
tion.Ifthissetup codeisstill per turbingthe results, the testing.B parameter provides met h-
odstostop, resume, and res et thetimer,but these are rarelyneeded.
No w that wehaveabenchmarkand tests, itseasy totry out ide as formak ingthe program
faster. Perhaps the most obviou s opt imizat ionistomake IsPalindromessecon d lo opstop
ch eck ing atthe midpoint, toavoid doingeachcomparisontwice:
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
322 CHAPTER 11. TESTING
n:=len(letters)/2
for i := 0; i < n; i++ {
if letters[i] != letters[len(letters)-1-i] {
return false
}
}
return true
But as is often the cas e,anobv iou s opt imizat iondoesntalways yield the exp ected benefit.
Thisone delivered a mere 4% imp rov ement inone exp eriment.
$gotest -bench=.
PASS
BenchmarkIsPalindrome-8 1000000 992 ns/op
ok gopl.io/ch11/word2 2.093s
Anot her ideaistopre-al locate a sufficientlylarge array for use by letters,rat her than
exp anditbysuccessive cal lsto append.Declaring letters as an array ofthe rig htsize,like
this,
letters := make([]rune, 0, len(s))
for _, r := range s {
if unicode.IsLetter(r) {
letters = append(letters, unicode.ToLower(r))
}
}
yields animp rov ement ofnearly35%, andthe benchmark runner now rep ortsthe average
ov er2,000,000 iterat ions.
$gotest -bench=.
PASS
BenchmarkIsPalindrome-8 2000000 697 ns/op
ok gopl.io/ch11/word2 1.468s
As thisexampleshows, the fastest program isoften the one thatmakes the fewestmemor y
al location s.The -benchmem command-line flag will include memor y al location statist ics in its
report.Herewecompare the numberofallocat ions beforethe opt imizat ion:
$gotest -bench=. -benchmem
PASS
BenchmarkIsPalindrome 1000000 1026 ns/op 304 B/op 4allocs/op
andafter it:
$gotest -bench=. -benchmem
PASS
BenchmarkIsPalindrome 2000000 807 ns/op 128 B/op 1 allocs/op
Cons olid atingthe allocat ions inasinglecal l to make eliminated 75% ofthe allocat ions and
halved the quant ity ofallocated memor y.
Benchmarks like thistel l us the abs olut e time required for a given operat ion, but inmanyset-
tingsthe int erest ing per for mance question s areabout the re lat ive timings oftwo dif ferent
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 11.5. PROFILING 323
op erat ions.For example, ifafunctiontakes 1mstopro cess1,000 elements, how lon g wi l l it
take topro cess10,000 oramillion? Suchcomparisons revealthe asy mpt otic growt h of the
runningtimeofthe function. Another example: whatisthe bestsize for anI/O buf fer?
Benchmarks ofapp lic ationthroughputoverarange ofsizes can helpuscho ose the smallest
buffer thatdeliverssat isfac tor y perfor mance.Athirdexample: whichalgor it hmper for msbest
foragiven job?Benchmarksthatevaluate two dif ferentalgor it hms onthe sameinp utdat a
canoften showthe strengt hsand weaknessesofeachone onimp ortantorrepresent ative
work loads.
Comp arat ive benchmarksare justregu lar code.The y typicallytakethe for m of a single
parameter izedfunction, cal le d from several Benchmark func tions wit h dif ferentvalues, like
this:
func benchmark(b *testing.B, size int) { /* ... */ }
func Benchmark10(b *testing.B) {benchmark(b, 10) }
func Benchmark100(b *testing.B) {benchmark(b, 100) }
func Benchmark1000(b *testing.B) { benchmark(b, 1000) }
Theparameter size,whichspecifies the size ofthe inp ut, varies acrossbenchmarksbut is
cons tantwit hin eachbenchmark.Resist the tempt ation touse the parameter b.N as the inp ut
size.Unlessyou int erpretitasaniterat ioncount for a xe d-size input, the results ofyour
benchmarkwill bemeaningless.
Patt ernsreveale d by comparat ive benchmarksare par tic ularlyusefuldur ingprogram design,
butwedontthrow the benchmarksaway whenthe program iswor king. Asthe program
evolves, orits inputgrows, oritisdeploye d on new operat ingsystems orpro cessors wit h dif-
ferent charac ter ist ics, wecan reuse those benchmarkstorevisit desig n de cisions.
Exercis e 11.6: Wr ite benchmarkstocompare the PopCount implementation inSec tion 2.6.2
with yoursolut ion s to Exercis e 2.4 andExercis e 2.5. Atwhatpoint doesthe table-b ased
approach bre akeven?
Exercis e 11.7: Wr ite benchmarksfor Add, UnionWith,and other met hodsof *IntSet (§6.5)
usinglarge pseudo-randominp uts. How fastcan you makethese met hodsrun? How doesthe
ch oice ofwordsize affe ctper for mance? How fastis IntSet comp are d to a set implementation
basedonthe bui lt-in map typ e?
11.5. Pro filing
Benchmarks areusefulfor measuring the per for mance ofspecificoperat ions,but whenwere
tr yingtomakeaslowprogram faster, weoften havenoide a wheretobeg in. Ev ery program-
merknows Don ald Knu thsaph orism aboutpremature opt imizat ion, whichapp eared in
‘‘St ruc tured Programmingwit h go toStatements’’ in 1974. Although often misinterprete d to
me anper for mance doesntmatter, inits originalcontext wecan discer n adif ferentmeaning:
Thereisnodou bt that the grai l of efficiency leads toabu se. Programmerswaste
enor mou s amountsoftimethin kingabout,orwor rying about,the speed ofnon critical
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
324 CHAPTER 11. TESTING
partsoftheir programs, andthese att emp tsatefficiency actuallyhaveastron g negat ive
impact when debuggingand maintenanceare con sidered.We sh oul d forgetabout
smal l efficiencies, say about97% ofthe time: premature opt imizat ionisthe rootofall
evil.
Ye tweshouldnot passupour opp ortunities in thatcritical3%. A go o d prog rammer
wi l l notbelulled int o comp lacency bysuchreasoning, hewill bewis e to lookcaref ully
at the criticalcode; but only af ter that code has beenidentied.Itisoften a mist ake to
make a prior i judg ments aboutwhatpar ts of a prog ram arereallycritical, since the
univers alexp erience ofprogrammerswho havebeenusingmeasurement tools has
been thattheir intuit ive guessesfai l.
Wh enwewishtolookcaref ullyatthe speed ofour programs, the besttechnique for identify-
ingthe criticalcodeis proling.Profilingisanaut omated appro ach to per for mance measure-
ment bas edonsamplinganumberofprofile events during exe cut ion,thenext rapol ating fro m
them dur ingapost-pro cessingstep; the resulting statist icalsummar y is cal le d a prole.
Go sup por tsmanykinds ofprofiling, eachcon cer ned wit h adif ferentasp ect ofper for mance,
butall oftheminv olverecordingasequence ofeventsofint erest,eachofwhichhas an accom-
pany ing stack tracethestack offunctioncal lsactiveatthe moment ofthe event.The
go test to olhas bui lt-in sup por t forseveral kinds ofprofiling.
A CPU prole identifies the functions whose exe cut ion requires the most CPU time. The cur-
rent lyrunningthreadoneachCPU isint err upt edper iodic ally bythe operat ingsystemevery
fe w mi l lisecond s,wit h each int err upt ion recordingone profile event beforenor mal exec ution
resumes.
A he approle identifies the statementsresponsible for allocat ingthe most memor y.The
profilinglibrar y samp les cal lstothe int ernal memor y al location routinessothatonaverage,
on e profile event isrecorde d per512KB ofallocated memor y.
A bl o cki n g prole identifies the operat ions responsible for blo cking goro utinesthe lon g est,
such assystemcal ls, channel sends and receives, andacquisition s of locks. Theprofiling
librar y re cords anevent every timeagoroutine isblo cke d by one ofthese operat ions.
Gat her ingaprofile for codeunder testisaseasy as enablingone ofthe flags below.Becaref ul
when usingmorethanone flag at a time, how ever: the machiner y forgat her ingone kindof
profile may ske w theresults ofothers.
$gotest -cpuprofile=cpu.out
$gotest -blockprofile=block.out
$gotest -memprofile=mem.out
Itseasy toadd profilingsup por t to non-testprogramstoo,thoug h thedet ails ofhow wedo
that varybet weenshort-live d command-linetools and lon g-r unningser ver applic ations.
Profilingisesp eci ally usefulinlon g-r unningapp lic ations,sothe Goruntimesprofiling
fe aturescan beenabled under programmer cont rol usingthe runtime API.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 11.5. PROFILING 325
Once weve gat hered a profile,weneed toanalyze itusingthe pprof to ol. Thisisastand ard
part ofthe Godistr ibution,but since itsnot aneverydaytool, itsaccessedindirec tly using
go tool pprof.Ithas dozensoffeaturesand opt ion s,but basic use requires onlytwo argu-
ments, the exe cut ablethatpro duce d theprofile andthe profile log.
To m akeprofilingefficientand tosavespace,the log does not include functionnames; instead,
func tions are identied bytheir addresses. Thismeans that pprof ne e dsthe exe cut ablein
order tomakesense ofthe log . Although go test usuallydis cards the testexe cut ableoncethe
test iscomplete, whenprofilingisenabled itsaves the exe cut ableas foo.test,where foo is
thenameofthe teste d package.
Thecommand s belowshowhow togat her anddispl ayasimpleCPU profile.Weve selec ted
on e of the benchmarksfro m the net/http package. Itisusu allybettertoprofile specific
benchmarks thathavebeencon str ucted toberepresent ative ofwor klo adsone cares about.
Benchmarking testcas es is almostnever represent ative , whichiswhy wedis abled themby
usingthe lt er -run=NONE.
$gotest -run=NONE -bench=ClientServerParallelTLS64 \
-cpuprofile=cpu.log net/http
PASS
BenchmarkClientServerParallelTLS64-8 1000
3141325 ns/op 143010 B/op 1747 allocs/op
ok net/http 3.395s
$gotool pprof -text -nodecount=10 ./http.test cpu.log
2570ms of 3590ms total (71.59%)
Dropped 129 nodes (cum <= 17.95ms)
Showing top 10 nodes out of 166 (cum >= 60ms)
flat flat% sum% cum cum%
1730ms 48.19% 48.19% 1750ms 48.75% crypto/elliptic.p256ReduceDegree
230ms 6.41% 54.60% 250ms 6.96% crypto/elliptic.p256Diff
120ms 3.34% 57.94% 120ms 3.34% math/big.addMulVVW
110ms 3.06% 61.00% 110ms 3.06% syscall.Syscall
90ms 2.51% 63.51% 1130ms 31.48% crypto/elliptic.p256Square
70ms 1.95% 65.46% 120ms 3.34% runtime.scanobject
60ms 1.67% 67.13% 830ms 23.12% crypto/elliptic.p256Mul
60ms 1.67% 68.80% 190ms 5.29% math/big.nat.montgomery
50ms 1.39% 70.19% 50ms 1.39% crypto/elliptic.p256ReduceCarry
50ms 1.39% 71.59% 60ms 1.67% crypto/elliptic.p256Sum
The -text flag specifies the out put for mat, in thiscas e,atextualtable wit h on e rowper func-
tion,sor ted sothe ‘‘hott est’’ func tionsthos e that cons ume the most CPU cyc lesappear
rs t.The -nodecount=10 flag limits the resultto10rows. For gross per for mance pro blems,
this textualfor mat may beenoug h to pinpointthe cause.
Thisprofile tel lsusthatellip tic-c urve crypt ography isimp ortanttothe per for mance ofthis
partic ularHTTPS benchmark.Bycontrast, if a profile isdominated bymemor y al location
func tions fro m the runtime package, reducingmemor y cons ump tionmay beawor thw hile
opt imizat ion.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
326 CHAPTER 11. TESTING
Fo r more subtlepro blems, you may bebetteroff usingone of pprofsgraphic al displays.
Thes e re quireGraphViz, whichcan bedow nlo ade d from www.graphviz.org.The -web flag
then rendersadirec ted graphofthe functions ofthe program, annot ate d by their CPU profile
numb ers and colored toindic atethe hott est functions.
Weve onlyscratch edthe sur face of Gosprofilingtools here. Tofind out more, readthe
‘‘ProfilingGoPrograms’’ ar ticle onthe GoBlog .
11.6. Example Functions
Thethirdkindoffunctiontre ate d sp eci ally by go test is anexamplefunction, one whose
name startswit h Example.Ithas neither parametersnor results. Heresanexamplefunction
for IsPalindrome:
func ExampleIsPalindrome() {
fmt.Println(IsPalindrome("A man, a plan, a canal: Panama"))
fmt.Println(IsPalindrome("palindrome"))
// Output:
// true
// false
}
Examplefunctions ser vethree pur pos es. Theprimary one isdocumentation:agood example
canbeamoresuccinct orint uit ive way toconve y thebeh avior of a librar y func tionthanits
pros e des crip tion, especi ally whenusedasareminder orquickreference.Anexamplecan
also demon stratethe int erac tionbet weenseveral typ es andfunctions belon gingtoone API,
whereasprose documentation mustalways beatt ach edtoone place,likeatyp e or function
de clarat ionorthe packageasawhole.And unlikeexamples wit hin comments, examplefunc-
tion s arerealGocode, subjec t to compi le-t imeche cking , so the y dontbecom e st ale as the
co de evolves.
Basedonthe suffixofthe Example func tion, the web-b ased documentation ser ver godoc
asso ciatesexamplefunctions wit h thefunctionorpackagethe y exemplif y,so ExampleIs-
Palindrome wouldbeshown with thedocumentation for the IsPalindrome func tion, andan
examplefunctioncal le d ju st Example wouldbeass oci ated wit h the word package as a whole.
Thesecon d purpos e is thatexamples areexe cut abletests run by go test.Ifthe examplefunc-
tion containsanal // Output: commentlikethe one above , thetestdriverwill exe cut e the
func tionand che ckthatwhatitprint edtoits stand ard out put match esthe text wit hin the
comment.
Thethirdpur pos e of anexampleishands-onexp erimentation.The godoc server at
golang.org us esthe GoPlayg round tolet the usereditand run eachexamplefunctionfro m
within a web brows er,asshown inFigure11.4. Thisisoften the fastest way toget a feelfor a
partic ularfunctionorlangu agefeature .
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 11.6. EXAMPLE FUNCTIONS 327
Figure 11.4. An int erac tiveexampleof strings.Join in godoc.
Thefinaltwo chapt ers ofthe bookexaminethe reflect and unsafe packages, whichfew Go
prog rammersregu larly useandevenfewer ne e d to use.Ifyou haventwritt enany subst ant ial
Go programsyet, now wou ldbeagood time to do that.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
This page intentionally left blank
From the Library of YIGUANG HU
ptg16105617
12
Re flection
Go provides a mechanism toupdatevar iables andins pec t their values at run time, to cal l their
methods, andtoapp l y theoperat ions int rinsic totheir represent ation,all wit houtknowing
their typ es at comp ile time. Thismechanism iscal le d reec tion.Reflec tion als o lets ustre at
typesthems elves as rs t-class values.
In thischapt er, well explore Gosreflec tion featurestosee how the y increase the expres-
siveness ofthe langu age, and inpar tic ularhow the y arecruci al to the imp lementation of two
importantAPIs: str ing for matting provide d by fmt,and pro tocol encodingprovide d by pack-
ages like encoding/json and encoding/xml.Reflec tion isals o essent ial tothe templ ate
me chanism provide d by the text/template and html/template packages wesaw in
Section4.6. How ever, reflec tion iscomplex toreasonabout and notfor casualuse,so
although these packages areimp lemente d usingreflec tion,the y do not exp osereflec tion in
their own APIs.
12.1. Why Reflection?
Sometimesweneed towrite a functioncap able ofdealingunifor mly wit h values oftyp es that
dontsat isf y acommonint erface,donthaveaknown represent ation,ordontexist atthe time
we desig n thefunctionor evenall three.
Afami liar exampleisthe for matting log ic within fmt.Fprintf,whichcan usefullyprint an
arbit rar y value ofany typ e,evenauser-define d on e.Letstry toimp lementafunctionlikeit
usingwhatweknowalready.For simplicity,our functionwill acceptone argumentand will
return theresult as a str ing like fmt.Sprint do es, so well cal l it Sprint.
We start wit h atyp e sw itch thattests whether the argumentdefinesaString method,and cal l
it ifso. Wethenadd swit c h casesthattestthe valuesdynamic typ e against eachofthe basic
329
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
330 CHAPTER 12. REFLECTION
typesstring, int, bool,and soonandper for m theappro priatefor matting operat ionin
each cas e.
func Sprint(x interface{}) string {
type stringer interface {
String() string
}
switch x := x.(type) {
case stringer:
return x.String()
case string:
return x
case int:
return strconv.Itoa(x)
// ...similar cases for int16, uint32, and so on...
case bool:
if x {
return "true"
}
return "false"
default:
// array, chan, func, map, pointer, slice, struct
return "???"
}
}
Buthow dowedealwit h ot her typ es, li ke []float64, map[string][]string,and soon? We
couldadd morecas es, butthe numberofsuchtyp es is infinite. And whatabout named typ es,
li ke url.Values?Evenifthe typ e sw itch had a cas e forits underly ing typ e
map[string][]string,itwou ldntmatch url.Values becaus e thetwo typ es arenot iden-
tical, and the typ e sw itch cannot include a cas e foreachtyp e li ke url.Values becaus e that
wouldrequirethislibrar y to dep enduponits clients.
Wi thout a way toins pec t therepresent ation of values ofunknown types, wequicklyget stuck.
Wh atweneed isreflec tion.
12.2. reflect.Type and reflect.Value
Reec tion isprovide d by the reflect package. Itdefinestwo imp ortanttyp es, Type and
Value.AType repres ents a Gotyp e.Itisanint erface wit h many met hodsfor dis criminat ing
amon g typesand ins pec ting their comp onents, like the elds ofastr uct orthe parametersofa
func tion. Thesoleimp lementation of reflect.Type is the typ e des crip tor (§7.5), the same
entity thatidentifies the dynamic typ e of anint erface value.
The reflect.TypeOf func tionaccepts any interface{} andretur nsits dynamic typ e as a
reflect.Type:
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 12.2. REFLECT.TYPE AND REFLECT.V ALUE331
t:=reflect.TypeOf(3) // areflect.Type
fmt.Println(t.String()) // "int"
fmt.Println(t) // "int"
The TypeOf(3) call above assig nsthe value 3 to the interface{} parameter.Recal l from
Section7.5 thatanassig nmentfro m acon crete value toanint erface typ e perfor msanimp licit
interface conv ersion, whichcre atesanint erface value consisting oftwo components: its
dy namic type is the operandstyp e (int)and its dy namic valu e is the operandsvalue (3).
Becaus e reflect.TypeOf returns an int erface valuesdynamic typ e,italways retur nsacon-
cretetyp e.So, for example, the codebelow prints "*os.File",not "io.Writer".Later,we
wi l l seethat reflect.Type is cap able ofrepresent ing int erface typ es to o.
var w io.Writer = os.Stdout
fmt.Println(reflect.TypeOf(w)) // "*os.File"
No tice that reflect.Type satisfies fmt.Stringer.Because print ing the dynamic typ e of an
interface value isusefulfor debug gingand log ging, fmt.Printf prov ides a shorthand, %T,that
us es reflect.TypeOf internal ly:
fmt.Printf("%T\n", 3) // "int"
Theother importanttyp e in the reflect packageis Value.Areflect.Value canholda
value ofany typ e.The reflect.ValueOf func tionaccepts any interface{} andretur nsa
reflect.Value cont ainingthe int erfacesdynamic value.Aswit h reflect.TypeOf,the
resu lts of reflect.ValueOf arealways concrete, but a reflect.Value canholdint erface val-
ues too.
v:=reflect.ValueOf(3) // a reflect.Value
fmt.Println(v) // "3"
fmt.Printf("%v\n", v) // "3"
fmt.Println(v.String()) // NOTE: "<int Value>"
Like reflect.Type, reflect.Value also sat isfies fmt.Stringer,but unlessthe Value holds
astr ing , theresultofthe String method reveals onlythe typ e.Ins tead, use the fmt packages
%v verb,whichtre ats reflect.Valuesspeci ally.
Callingthe Type method onaValue returnsits typ e as a reflect.Type:
t:=v.Type() // areflect.Type
fmt.Println(t.String()) // "int"
Theinv ers e op erat ionto reflect.ValueOf is the reflect.Value.Interface method.It
returnsan interface{} holdingthe samecon crete value as the reflect.Value:
v:=reflect.ValueOf(3) // a reflect.Value
x:=v.Interface() // an interface{}
i:=x.(int) // an int
fmt.Printf("%d\n", i) // "3"
A reflect.Value andan interface{} canbot h hold arbit rar y values. Thedif ference isthat
an empt y interface hides the represent ation and int rinsic operat ions ofthe value itholds and
exp oses non e of itsmet hods, sounlessweknowits dynamic typ e anduse a typ e assertionto
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
332 CHAPTER 12. REFLECTION
peer inside it(as wedid abov e), there islit tle wecan do tothe value wit hin. Incontrast, a
Value hasmanymet hodsfor ins pec ting its cont ents, regardlessofits typ e.Letsuse themfor
oursecon d attemp t at a general for matting function, whichwell cal l format.Any.
In ste adofatyp e sw itch,weuse reflect.Values Kind method todis criminatethe cas es.
Although there are infinite lymanytyp es, thereare onlyafinite numberof ki nds of typ e: the
basic typ es Bool, String,and all the numbers;the aggregatetyp es Array and Struct;the ref-
erence typ es Chan, Func, Ptr, Slice,and Map; Interface types; and nally Invalid,meaning
no value at all. (Thezerovalue ofareflect.Value haskind Invalid.)
gopl.io/ch12/format
package format
import (
"reflect"
"strconv"
)
// Any formats any value as a string.
func Any(value interface{}) string {
return formatAtom(reflect.ValueOf(value))
}
// formatAtom formats a value without inspecting its internal structure.
func formatAtom(v reflect.Value) string {
switch v.Kind() {
case reflect.Invalid:
return "invalid"
case reflect.Int, reflect.Int8, reflect.Int16,
reflect.Int32, reflect.Int64:
return strconv.FormatInt(v.Int(), 10)
case reflect.Uint, reflect.Uint8, reflect.Uint16,
reflect.Uint32, reflect.Uint64, reflect.Uintptr:
return strconv.FormatUint(v.Uint(), 10)
// ...floating-point and complex cases omitted for brevity...
case reflect.Bool:
return strconv.FormatBool(v.Bool())
case reflect.String:
return strconv.Quote(v.String())
case reflect.Chan, reflect.Func, reflect.Ptr, reflect.Slice, reflect.Map:
return v.Type().String() + " 0x" +
strconv.FormatUint(uint64(v.Pointer()), 16)
default: // reflect.Array, reflect.Struct, reflect.Interface
return v.Type().String() + " value"
}
}
So far,our functiontre ats eachvalue as an indivisiblethingwit h no int ernal str ucturehence
formatAtom.For aggregatetyp es (str uctsand arrays) andint erfaces itprintsonlythe ty pe of
thevalue,and for reference typ es (channel s,functions,point ers,slices, andmaps), itprintsthe
type and the reference addressinhexade cimal.Thisislessthanide al butstill a maj or
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 12.3. DISPLAY,ARECURSIVE VALUE PRINTER 333
improv ement,and since Kind is con cer ned onlywit h theunderly ing represent ation, for-
mat.Any worksfor named typ es to o.For example:
var x int64 = 1
var d time.Duration = 1 * time.Nanosecond
fmt.Println(format.Any(x)) // "1"
fmt.Println(format.Any(d)) // "1"
fmt.Println(format.Any([]int64{x})) // "[]int64 0x8202b87b0"
fmt.Println(format.Any([]time.Duration{d})) // "[]time.Duration 0x8202b87e0"
12.3. Display,aRecursiveValue Printer
Next well takealookathow toimp rov e thedispl ayofcomposite typ es. Rather thantry to
copy fmt.Sprint exac tly,well bui ld adebug gingutilit y func tioncal le d Display that, given
an arbit rar ily complex value x,printsthe completestr uctureofthatvalue,lab elingeachele-
ment wit h thepat h by whichitwas found.Letsstart wit h an example.
e, _ := eval.Parse("sqrt(A / pi)")
Display("e", e)
In the cal l ab ove , theargumentto Display is a syntaxtre e from the expressionevaluatorin
Section7.9. Theout put of Display is shown below :
Display e (eval.call):
e.fn = "sqrt"
e.args[0].type = eval.binary
e.args[0].value.op = 47
e.args[0].value.x.type = eval.Var
e.args[0].value.x.value = "A"
e.args[0].value.y.type = eval.Var
e.args[0].value.y.value = "pi"
Wh ere possible, you shouldavoid exposingreflec tion inthe API ofapackage. Well define an
unexp orted function display to dothe realwor k of the rec ursion, andexp ort Display,asim-
plewrapperaro und itthataccepts an interface{} parameter :
gopl.io/ch12/display
func Display(name string, x interface{}) {
fmt.Printf("Display %s (%T):\n", name, x)
display(name, reflect.ValueOf(x))
}
In display,well use the formatAtom func tionwedefine d earlier toprint elementary values
basic typ es, func tions,and channel sbutwell use the methodsof reflect.Value to rec ur-
sive lydispl ayeachcomponent ofamorecomplex typ e.Asthe rec ursiondes cends,the path
st ring, whichinitial lydes crib esthe startingvalue (for ins tance, "e"), will beaug mente d to
indic atehow wereach edthe cur rentvalue (for ins tance, "e.args[0].value").
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
334 CHAPTER 12. REFLECTION
Sincewere nolon g erpretendingtoimp lement fmt.Sprint,wewill use the fmt packageto
ke epour exampleshort.
func display(path string, v reflect.Value) {
switch v.Kind() {
case reflect.Invalid:
fmt.Printf("%s = invalid\n", path)
case reflect.Slice, reflect.Array:
for i := 0; i < v.Len(); i++ {
display(fmt.Sprintf("%s[%d]", path, i), v.Index(i))
}
case reflect.Struct:
for i := 0; i < v.NumField(); i++ {
fieldPath := fmt.Sprintf("%s.%s", path, v.Type().Field(i).Name)
display(fieldPath, v.Field(i))
}
case reflect.Map:
for _, key := range v.MapKeys() {
display(fmt.Sprintf("%s[%s]", path,
formatAtom(key)), v.MapIndex(key))
}
case reflect.Ptr:
if v.IsNil() {
fmt.Printf("%s = nil\n", path)
}else {
display(fmt.Sprintf("(*%s)", path), v.Elem())
}
case reflect.Interface:
if v.IsNil() {
fmt.Printf("%s = nil\n", path)
}else {
fmt.Printf("%s.type = %s\n", path, v.Elem().Type())
display(path+".value", v.Elem())
}
default: // basic types, channels, funcs
fmt.Printf("%s = %s\n", path, formatAtom(v))
}
}
Letsdis cussthe cas es in order.
Slices and array s: Thelog ic is the samefor bot h.The Len method retur nsthe numberofele-
mentsofaslice orarray value,and Index(i) retr ieves the element at index i,als o as a
reflect.Value;itpanics if i is out of bound s.These are analogou s to the bui lt-in len(a)
and a[i] op erat ions onsequences. The display func tionrec ursivelyinv okesits elf oneach
elementofthe sequence,app endingthe subscript not ation "[i]" to the pat h.
Although reflect.Value hasmanymet hods, onlyafew are safetocal l on any given value.
Fo r example, the Index method may becal le d on values ofkind Slice, Array,or String,but
panics for any other kind.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 12.3. DISPLAY,ARECURSIVE VALUE PRINTER 335
Structs: The NumField method rep ortsthe numberoffields inthe str uct,and Field(i)
returnsthe value ofthe i-t h eldasareflect.Value.The listoffields includes ones
prom ote d from anony mou s elds.Toapp end the eldselec tor not ation ".f" to the pat h, we
mu stobt ain the reflect.Type of the str uct and accessthe nameofits i-t h eld.
Ma ps: The MapKeys method retur nsaslice of reflect.Values, one per map key.Asusu al
when iterat ingoveramap,the order isundefine d. MapIndex(key) returnsthe value cor-
resp ondingto key.Weapp end the subscript not ation "[key]" to the pat h. (Were cut tinga
corner here. The typ e of a map key isntrestr icted tothe typ es formatAtom hand les best;
ar rays, st ruc ts, andint erfaces can also bevalid map keys. Extendingthiscas e to print the key
in full isExercis e 12.1.)
Po int ers: The Elem method retur nsthe var iable point edtobyapoint er, again as a
reflect.Value.Thisoperat ionwou ldbesafeevenifthe point ervalue is nil,inwhichcas e
theresultwou ldhavekind Invalid,but weuse IsNil to detec t ni l pointers explicitlysowe
canprint a moreappro priatemessage . We prexthe pat h with a "*" andparenthesize itto
avoidambiguity.
In ter fac es: Again, weuse IsNil to test whether the int erface isnil,and ifnot,weret rie veits
dy namic value using v.Elem() andprint its typ e andvalue.
No w that our Display func tioniscomplete, letsput it towor k.The Movie type belowisa
slig htvar iat iononthe one inSec tion 4.5:
type Movie struct {
Title, Subtitle string
Year int
Color bool
Actor map[string]string
Oscars []string
Sequel *string
}
Letsdeclare a value ofthistyp e andsee what Display do es with it:
strangelove := Movie{
Title: "Dr. Strangelove",
Subtitle: "How I Learned to Stop Worrying and Love the Bomb",
Year: 1964,
Color: false,
Actor: map[string]string{
"Dr. Strangelove": "Peter Sellers",
"Grp. Capt. Lionel Mandrake": "Peter Sellers",
"Pres. Merkin Muffley": "Peter Sellers",
"Gen. Buck Turgidson": "George C. Scott",
"Brig. Gen. Jack D. Ripper": "Sterling Hayden",
`Maj. T.J. "King" Kong`:"Slim Pickens",
},
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
336 CHAPTER 12. REFLECTION
Oscars: []string{
"Best Actor (Nomin.)",
"Best Adapted Screenplay (Nomin.)",
"Best Director (Nomin.)",
"Best Picture (Nomin.)",
},
}
Thecal l Display("strangelove", strangelove) pr ints:
Display strangelove (display.Movie):
strangelove.Title = "Dr. Strangelove"
strangelove.Subtitle = "How I Learned to Stop Worrying and Love the Bomb"
strangelove.Year = 1964
strangelove.Color = false
strangelove.Actor["Gen. Buck Turgidson"] = "George C. Scott"
strangelove.Actor["Brig. Gen. Jack D. Ripper"] = "Sterling Hayden"
strangelove.Actor["Maj. T.J. \"King\" Kong"] = "Slim Pickens"
strangelove.Actor["Dr. Strangelove"] = "Peter Sellers"
strangelove.Actor["Grp. Capt. Lionel Mandrake"] = "Peter Sellers"
strangelove.Actor["Pres. Merkin Muffley"] = "Peter Sellers"
strangelove.Oscars[0] = "Best Actor (Nomin.)"
strangelove.Oscars[1] = "Best Adapted Screenplay (Nomin.)"
strangelove.Oscars[2] = "Best Director (Nomin.)"
strangelove.Oscars[3] = "Best Picture (Nomin.)"
strangelove.Sequel = nil
We can use Display to displ aythe int ernalsoflibrar y types, suchas *os.File:
Display("os.Stderr", os.Stderr)
// Output:
// Display os.Stderr (*os.File):
// (*(*os.Stderr).file).fd = 2
// (*(*os.Stderr).file).name = "/dev/stderr"
// (*(*os.Stderr).file).nepipe = 0
No tice thatevenunexp orted elds are visible toreflec tion.Bewarethatthe par tic ularout put
of thisexamplemay varyacrossplatfor msand may change overtimeaslibrar ies evolve.
(Thos e elds are privatefor a reason!) Wecan evenapp l y Display to a reflect.Value and
watch ittraverse the int ernal represent ation of the typ e des crip tor for *os.File.The out put
of the cal l Display("rV", reflect.ValueOf(os.Stderr)) is shown below,thoug h of cours e
your mile agemay vary:
Display rV (reflect.Value):
(*rV.typ).size = 8
(*rV.typ).hash = 871609668
(*rV.typ).align = 8
(*rV.typ).fieldAlign = 8
(*rV.typ).kind = 22
(*(*rV.typ).string) = "*os.File"
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 12.3. DISPLAY,ARECURSIVE VALUE PRINTER 337
(*(*(*rV.typ).uncommonType).methods[0].name) = "Chdir"
(*(*(*(*rV.typ).uncommonType).methods[0].mtyp).string) = "func() error"
(*(*(*(*rV.typ).uncommonType).methods[0].typ).string) = "func(*os.File) error"
...
Observethe dif ference bet weenthese two examples:
var i interface{} = 3
Display("i", i)
// Output:
// Display i (int):
// i = 3
Display("&i", &i)
// Output:
// Display &i (*interface {}):
// (*&i).type = int
// (*&i).value = 3
In the rs t example, Display calls reflect.ValueOf(i),whichretur nsavalue ofkind Int.
As wemention edinSec tion 12.2, reflect.ValueOf always retur nsaValue of a con crete typ e
since itext racts the contents of an int erface value.
In the secon d example, Display calls reflect.ValueOf(&i),whichretur nsapoint erto i,of
kind Ptr.The switch cas e for Ptr calls Elem on thisvalue,whichretur nsaValue repres enting
the variab le i itself,ofkind Interface.AValue obtained indirec tly,likethisone,may rep-
resent any value at all, includingint erfaces. The display func tioncal lsits elf rec ursivelyand
this time, itprintssep aratecomponentsfor the int erfacesdynamic typ e andvalue.
As cur rentlyimp lemente d, Display wi l l ne ver ter minateifitencount ers a cyc leinthe obj e ct
graph, such as thislin ked listthateats its own tai l:
// a struct that points to itself
type Cycle struct{ Value int; Tail *Cycle }
var c Cycle
c=Cycle{42, &c}
Display("c", c)
Display pr intsthisever-grow ing exp ansion:
Display c (display.Cycle):
c.Value = 42
(*c.Tail).Value = 42
(*(*c.Tail).Tail).Value = 42
(*(*(*c.Tail).Tail).Tail).Value = 42
...
ad infinitum
...
Many Goprogramscontain at leastsom e cyclic dat a. Making Display ro bustagainst such
cycles istrick y,requir ingaddition albookkeeping torecordthe set ofreferences thathavebeen
fo llowe d so far ; it iscostlytoo.Ageneral solut ion requires unsafe language features, as we
wi l l see in Sec tion 13.3.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
338 CHAPTER 12. REFLECTION
Cy cles pos e lessofapro blem for fmt.Sprint becaus e it rarelytries toprint the complete
st ruc ture. For example, whenitencount ers a point er, itbre aks the rec ursionbyprint ing the
pointersnumer ic value.Itcan get stuck tryingtoprint a slice ormap thatcontainsits elf as an
element, but suchrarecas es do not war rantthe con siderable extra tro ubleofhandlingcyc les.
Exercis e 12.1: Extend Display so thatitcan displaymaps whose keysare str uctsorarrays.
Exercis e 12.2: Make display safe touse oncyc lic dat a st ruc tures byboundingthe numberof
steps ittakes beforeabandoningthe rec ursion. (InSec tion 13.3, well see another way to
detec t cycles.)
12.4. Example: Encoding S-Expressions
Display is a debug gingroutine for displ aying str uctured dat a, butitsnot far short ofbeing
able toencodeor marshal arbit rar y Go obj e cts as messagesinapor table not ation suitablefor
inter-processcommunic ation.
As wesaw in Sec tion 4.5, Gosstand ard librar y supp ortsavar ietyoffor mats, includingJSON,
XML, andASN.1. Another not ation thatisstill widely usedis S-expre ssi ons,the syntaxof
Lisp.Unlikethe other not ation s,S-expressions are not sup por ted bythe Gostand ard librar y,
notleast because the y have nounivers allyaccepte d definition,despit e several attemp tsatstan-
dardizat ionand the existence ofmanyimp lementation s.
In thissec tion,well define a packagethatencodes arbit rar y Go obj e cts usinganS-expression
notation thatsup por tsthe fol low ing con str ucts:
42 integer
"hello" string (with Go-style quotation)
foo symbol (an unquoted name)
(1 2 3) list (zero or more items enclosed in parentheses)
Booleans are tradition allyencoded usingthe symbol t fortrue, and the emp tylist () or the
sy mbol nil forfalse,but for simplicity,our implementation ignores them. Itals o ig nores
ch annel s andfunctions,since their state isopaquetoreflec tion.And itignores realand com-
plex floating-p ointnumbers and int erfaces. Addingsup por t forthemisExercis e 12.3.
Well encodethe typ es of GousingS-expressions asfol lows. Int egers and str ingsare encoded
in the obv iou s way.Nil values areencoded asthe symbol nil.Arrays andslices areencoded
usinglistnot ation.
St ruc ts areencoded asalistoffieldbindings, each eldbindingbeingatwo-elementlist
whos e rs t element(asymbol) isthe eldnameand whose second elementisthe eldvalue.
Maps too are encoded asalistofpairs, wit h each pair beingthe key and value ofone map
entr y.Tradition ally, S-expressions represent lists ofkey/value pairsusingasingle cons cell
(key . value) foreachpair,rat her thanatwo-elementlist, but tosimplif y thedecodingwell
ig noredot ted listnot ation.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 12.4. EXAMPLE: ENCODING S-EXPRESSIONS 339
Enco dingisdon e by a singlerec ursivefunction, encode,shown below.Its str uctureisess en-
tial lythe same as thatof Display in the pre vious sec tion:
gopl.io/ch12/sexpr
func encode(buf *bytes.Buffer, v reflect.Value) error {
switch v.Kind() {
case reflect.Invalid:
buf.WriteString("nil")
case reflect.Int, reflect.Int8, reflect.Int16,
reflect.Int32, reflect.Int64:
fmt.Fprintf(buf, "%d", v.Int())
case reflect.Uint, reflect.Uint8, reflect.Uint16,
reflect.Uint32, reflect.Uint64, reflect.Uintptr:
fmt.Fprintf(buf, "%d", v.Uint())
case reflect.String:
fmt.Fprintf(buf, "%q", v.String())
case reflect.Ptr:
return encode(buf, v.Elem())
case reflect.Array, reflect.Slice: // (value ...)
buf.WriteByte('(')
for i := 0; i < v.Len(); i++ {
if i > 0 {
buf.WriteByte('')
}
if err := encode(buf, v.Index(i)); err != nil {
return err
}
}
buf.WriteByte(')')
case reflect.Struct: // ((name value) ...)
buf.WriteByte('(')
for i := 0; i < v.NumField(); i++ {
if i > 0 {
buf.WriteByte('')
}
fmt.Fprintf(buf, "(%s ", v.Type().Field(i).Name)
if err := encode(buf, v.Field(i)); err != nil {
return err
}
buf.WriteByte(')')
}
buf.WriteByte(')')
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
340 CHAPTER 12. REFLECTION
case reflect.Map: // ((key value) ...)
buf.WriteByte('(')
for i, key := range v.MapKeys() {
if i > 0 {
buf.WriteByte('')
}
buf.WriteByte('(')
if err := encode(buf, key); err != nil {
return err
}
buf.WriteByte('')
if err := encode(buf, v.MapIndex(key)); err != nil {
return err
}
buf.WriteByte(')')
}
buf.WriteByte(')')
default: // float, complex, bool, chan, func, interface
return fmt.Errorf("unsupported type: %s", v.Type())
}
return nil
}
The Marshal func tionwraps the encoder in an APIsimi lar tothose ofthe other encod-
ing/... packages:
// Marshal encodes a Go value in S-expression form.
func Marshal(v interface{}) ([]byte, error) {
var buf bytes.Buffer
if err := encode(&buf, reflect.ValueOf(v)); err != nil {
return nil, err
}
return buf.Bytes(), nil
}
Heresthe out put of Marshal applie d to the strangelove var iable fro m Section12.3:
((Title "Dr. Strangelove") (Subtitle "How I Learned to Stop Worrying and Lo
ve the Bomb") (Year 1964) (Actor (("Grp. Capt. Lionel Mandrake" "Peter Sell
ers") ("Pres. Merkin Muffley" "Peter Sellers") ("Gen. Buck Turgidson" "Geor
ge C. Scott") ("Brig. Gen. Jack D. Ripper" "Sterling Hayden") ("Maj. T.J. \
"King\" Kong" "Slim Pickens") ("Dr. Strangelove" "Peter Sellers"))) (Oscars
("Best Actor (Nomin.)" "Best Adapted Screenplay (Nomin.)" "Best Director (N
omin.)" "Best Picture (Nomin.)")) (Sequel nil))
Thewhole out put app earsonone lon g linewit h minimal spaces, mak ingithardtoread.
Heresthe sameout put manuallyfor matte d accordingtoS-expressionconvent ion s.Writing a
pretty-print erfor S-expressions islef t as a (challenging)exercis e;the dow nlo ad from gopl.io
includes a simpleversion.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 12.5. SETTING VARIABLES WITH REFLECT.V ALUE341
((Title "Dr. Strangelove")
(Subtitle "How I Learned to Stop Worrying and Love the Bomb")
(Year 1964)
(Actor (("Grp. Capt. Lionel Mandrake" "Peter Sellers")
("Pres. Merkin Muffley" "Peter Sellers")
("Gen. Buck Turgidson" "George C. Scott")
("Brig. Gen. Jack D. Ripper" "Sterling Hayden")
("Maj. T.J. \"King\" Kong" "Slim Pickens")
("Dr. Strangelove" "Peter Sellers")))
(Oscars ("Best Actor (Nomin.)"
"Best Adapted Screenplay (Nomin.)"
"Best Director (Nomin.)"
"Best Picture (Nomin.)"))
(Sequel nil))
Like the fmt.Print, json.Marshal,and Display func tions, sexpr.Marshal wi l l lo opforever
if cal le d with cyc lic dat a.
In Sec tion 12.6, well sketch out the imp lementation of the cor respondingS-expressiondecod-
ingfunction, but beforeweget there , well rs t ne e d to underst and howreflec tion can beused
to updateprogram variables.
Exercis e 12.3: Implementthe missingcas es of the encode func tion. Enco de boole ansas t and
nil,floating-p ointnumbers usingGosnot ation,and complex numbers like1+2i as
#C(1.0 2.0).Int erfaces can beencoded asa pair ofatyp e name and a value,for ins tance
("[]int" (1 2 3)),but bewarethatthisnot ation isambiguous:the reflect.Type.String
method may retur n thesamestr ing for dif ferenttyp es.
Exercis e 12.4: Mo dif y encode to prett y-print the S-expression in the sty leshown above .
Exercis e 12.5: Ad apt encode to emitJSONins teadofS-expressions.Testyourencoder using
thestand ard decoder, json.Unmarshal.
Exercis e 12.6: Ad apt encode so that, as an opt imizat ion, itdoesnot encodeaeldwhose
value isthe zerovalue ofits typ e.
Exercis e 12.7: Create a streamingAPI for the S-expressiondecoder,fol low ing the sty leof
json.Decoder (§4.5).
12.5. Setting Variables with reflect.Value
So far,reflec tion has only interpre ted values in our program in various ways. Thepoint ofthis
section, how ever, isto ch ange them.
Re call thatsom e Go expressions like x, x.f[1],and *p denot e var iables, but otherslike x+1
and f(2) do not.Avar iable isan ad dre ssabl e storagelocat ionthatcontainsavalue,and its
value may beupdated through thataddress.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
342 CHAPTER 12. REFLECTION
Asimi lar dist inc tionapp lies to reflect.Values. Som e areaddressable; othersare not.
Consider the fol low ing declarat ions:
x:=2 // value type variable?
a:=reflect.ValueOf(2) // 2int no
b:=reflect.ValueOf(x) // 2int no
c:=reflect.ValueOf(&x) // &x *int no
d:=c.Elem() // 2int yes (x)
Thevalue wit hin a is not addressable. Itismerelyacopyofthe int eger2.The sameistrueof
b.The value wit hin c is als o non-addressable, beingacopyofthe point ervalue &x.Infac t, no
reflect.Value returned by reflect.ValueOf(x) is addressable. But d,der ive d from c by
dereferencingthe point erwit hin it, referstoavar iable andisthu s addressable. Wecan use
this appro ach,cal ling reflect.ValueOf(&x).Elem(),toobt ain an addressable Value forany
var iable x.
We can askareflect.Value whet her itisaddressablethrough its CanAddr method:
fmt.Println(a.CanAddr()) // "false"
fmt.Println(b.CanAddr()) // "false"
fmt.Println(c.CanAddr()) // "false"
fmt.Println(d.CanAddr()) // "true"
We obt ain an addressable reflect.Value whenever weindirec t thro ugh a point er, evenifwe
st arted fro m anon-addressable Value.All the usu alrules for addressabi lit y have analogs for
reec tion.For example, since the slice indexingexpression e[i] implicitlyfol lowsapoint er, it
is addressableevenifthe expression e is not.Byanalog y, reflect.ValueOf(e).Index(i)
refers to a var iable,and isthu s addressableevenif reflect.ValueOf(e) is not.
To recov erthe var iable fro m an addressable reflect.Value re quires three steps. First,wecal l
Addr(),whichretur nsaValue holdingapointertothe var iable.Next, wecal l Interface()
on this Value,whichretur nsan interface{} value cont ainingthe point er. Final ly, ifwe
know the typ e of the var iable,wecan use a typ e assertiontoret rie vethe contentsofthe int er-
face as an ordinar y pointer. Wecan thenupdatethe var iable through the point er:
x:=2
d:=reflect.ValueOf(&x).Elem() // drefers to the variable x
px := d.Addr().Interface().(*int) // px := &x
*px = 3 // x = 3
fmt.Println(x) // "3"
Or,wecan updatethe var iable refer red tobyanaddressable reflect.Value direc tly,wit hout
usingapoint er, bycal lingthe reflect.Value.Set method:
d.Set(reflect.ValueOf(4))
fmt.Println(x) // "4"
Thesameche cks for assig nabilit y that areordinar ily per for med bythe compi ler aredon e at
runtimebythe Set methods. Above , thevar iable andthe value bot h have typ e int,but ifthe
var iable had beenan int64,the program wou ldpanic, soitscruci al to makesurethe value is
assig nable tothe typ e of the var iable:
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 12.5. SETTING VARIABLES WITH REFLECT.V ALUE343
d.Set(reflect.ValueOf(int64(5))) // panic: int64 is not assignable to int
Andofcours e calling Set on a non-addressable reflect.Value panics too:
x:=2
b:=reflect.ValueOf(x)
b.Set(reflect.ValueOf(3)) // panic: Set using unaddressable value
Thereare var iants of Set sp eci alizedfor cer tain gro ups ofbasic typ es: SetInt, SetUint, Set-
String, SetFloat,and soon:
d:=reflect.ValueOf(&x).Elem()
d.SetInt(3)
fmt.Println(x) // "3"
In som e ways these met hodsare morefor giv ing . SetInt,for example, will succe e d so lon g as
thevar iablestyp e is som e kind ofsig ned int eger, orevenanamed typ e whos e underly ing typ e
is a sig ned int eger, and ifthe value istoo large itwill bequiet lytrunc ate d to fit. But tre adcare-
fully: cal ling SetInt on a reflect.Value that referstoan interface{} var iable will panic,
even thoug h Set wouldsucce e d.
x:=1
rx := reflect.ValueOf(&x).Elem()
rx.SetInt(2) // OK, x = 2
rx.Set(reflect.ValueOf(3)) // OK, x = 3
rx.SetString("hello") // panic: string is not assignable to int
rx.Set(reflect.ValueOf("hello")) // panic: string is not assignable to int
var y interface{}
ry := reflect.ValueOf(&y).Elem()
ry.SetInt(2) // panic: SetInt called on interface Value
ry.Set(reflect.ValueOf(3)) // OK, y = int(3)
ry.SetString("hello") // panic: SetString called on interface Value
ry.Set(reflect.ValueOf("hello")) // OK, y = "hello"
Wh enweapp lie d Display to os.Stdout,wefound thatreflec tion can readthe values of
unexp orted str uct elds thatare inaccessibleaccordingtothe usu alrules ofthe langu age, like
the fd int eldofan os.File st ruc t on a Unix-li keplatfor m.How ever, reflec tion cannot
up datesuchvalues:
stdout := reflect.ValueOf(os.Stdout).Elem() // *os.Stdout, an os.File var
fmt.Println(stdout.Type()) // "os.File"
fd := stdout.FieldByName("fd")
fmt.Println(fd.Int()) // "1"
fd.SetInt(2) // panic: unexported field
An addressable reflect.Value re cords whether itwas obt ained bytraversinganunexp orted
st ruc t eldand,ifso, dis allowsmodification.Con sequently, CanAddr is not usu allythe rig ht
ch eck to use beforesetting a var iable.The rel ate d method CanSet reportswhether a
reflect.Value is addressable an d sett able:
fmt.Println(fd.CanAddr(), fd.CanSet()) // "true false"
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
344 CHAPTER 12. REFLECTION
12.6. Example: Decoding S-Expressions
Fo r each Marshal func tionprovide d by the stand ard librar ys encoding/... packages, there is
acor responding Unmarshal func tionthatdoesdecoding. For example, aswesaw in
Section4.5, given a byteslice cont ainingJSON-enco deddat a forour Movie type (§12.3), we
candecodeitlikethis:
data := []byte{/* ... */}
var movie Movie
err := json.Unmarshal(data, &movie)
The Unmarshal func tionusesreflec tion tomodif y thefields ofthe exist ing movie var iable,
creating new maps, str ucts, andslices as deter mined bythe typ e Movie andthe content ofthe
incomingdat a.
Letsnow imp lementasimple Unmarshal func tionfor S-expressions,analogou s to the stan-
dard json.Unmarshal func tionusedabove , andthe inv ers e of our earlier sexpr.Marshal.
We mustcaution you thatarobustand general implementation requires subst ant ial lymore
co de than will comfortably fit in thisexample, whichisalready lon g,sowehavetaken many
shortcuts. Wesup por t on lyalimite d su bsetofS-expressions and donot handleerror s grace-
fully. The codeisint ended toillustratereflec tion,not parsing.
Thelexer usesthe Scanner type fro m the text/scanner packagetobre akaninp utstream
into a sequence oftokenssuchascomments, identifiers, str ing lit erals, andnumer ic literals.
Thescanners Scan method advances the scanner andretur nsthe kindofthe next token,
whichhas typ e rune.Mosttokens, like '(',con sistofasinglerune, but the text/scanner
packagerepresentsthe kinds ofthe multi-charac ter tokens Ident, String,and Int using
smal l negat ive values oftyp e rune.Fol low ing a cal l to Scan that retur nsone ofthese kinds of
token, the scanners TokenText method retur nsthe text ofthe token.
Sinceatypicalparsermay need toins pec t thecur renttoken several times, but the Scan
method advances the scanner,wewrap the scanner in a helpertyp e called lexer that keeps
trackofthe token mostrecentlyretur ned by Scan.
gopl.io/ch12/sexpr
type lexer struct {
scan scanner.Scanner
token rune // the current token
}
func (lex *lexer) next() {lex.token = lex.scan.Scan() }
func (lex *lexer) text() string { return lex.scan.TokenText() }
func (lex *lexer) consume(want rune) {
if lex.token != want { // NOTE: Not an example of good error handling.
panic(fmt.Sprintf("got %q, want %q", lex.text(), want))
}
lex.next()
}
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 12.6. EXAMPLE: DECODING S-EXPRESSIONS 345
No w letstur n to the parser. Itcon sists oftwo princip alfunctions.The rs t of these, read,
re ads the S-expressionthatstartswit h thecur renttoken andupdates the var iable refer red to
by the addressable reflect.Value v.
func read(lex *lexer, v reflect.Value) {
switch lex.token {
case scanner.Ident:
// The only valid identifiers are
// "nil" and struct field names.
if lex.text() == "nil" {
v.Set(reflect.Zero(v.Type()))
lex.next()
return
}
case scanner.String:
s, _ := strconv.Unquote(lex.text()) // NOTE: ignoring errors
v.SetString(s)
lex.next()
return
case scanner.Int:
i, _ := strconv.Atoi(lex.text()) // NOTE: ignoring errors
v.SetInt(int64(i))
lex.next()
return
case '(':
lex.next()
readList(lex, v)
lex.next() // consume ')'
return
}
panic(fmt.Sprintf("unexpected token %q", lex.text()))
}
OurS-expressions use identifiersfor two distinc t purpos es, st ruc t eldnames andthe nil
value for a point er. The read func tiononlyhandles the lattercas e.Whenitencount ers the
scanner.Ident "nil",itsets v to the zerovalue ofits typ e usingthe reflect.Zero func tion.
Fo r anyother identifier,itrep ortsanerror.The readList func tion, whichwell see ina
moment,handles identifiersused as str uct eldnames.
A '(' tokenindic ates the start ofalist. Thesecon d func tion, readList,decodes a list int o a
var iable ofcomposite typ eamap,str uct,slice,orarraydep endingonwhatkindofGo
var iable were cur rentlypopu lat ing. Ineachcas e,the loopkeeps parsingitems unt i l it encoun-
ters the matchingclos e parent hesis, ')', as detec ted bythe endList func tion.
Theint erest ing par t is the rec ursion. Thesimplestcas e is anarray.Unt i l theclosing ')' is
seen, weuse Index to obt ain the var iable for eacharray elementand makearec ursivecal l to
read to popu lateit. Asinmanyother erro r cases, if the inp utdat a caus esthe decoder toindex
beyond the end ofthe array,the decoder panics. A simi lar approach isusedfor slices, except
we mustcre ate a new var iable for eachelement, popu lateit, thenapp end ittothe slice.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
346 CHAPTER 12. REFLECTION
Theloops for str uctsand maps mustparse a (key value) su blist oneachiterat ion. For
st ruc ts, thekey isasymbolidentifyingthe eld. Analogou s to the cas e forarrays, weobt ain
theexist ing var iable for the str uct eldusing FieldByName andmakearec ursivecal l to
popu lateit. For maps, the key may beofany typ e,and analogou s to the cas e forslices, wecre-
ateanew var iable,rec ursivelypopu lateit, and nallyins ert the newkey/value pair into the
map.
func readList(lex *lexer, v reflect.Value) {
switch v.Kind() {
case reflect.Array: // (item ...)
for i := 0; !endList(lex); i++ {
read(lex, v.Index(i))
}
case reflect.Slice: // (item ...)
for !endList(lex) {
item := reflect.New(v.Type().Elem()).Elem()
read(lex, item)
v.Set(reflect.Append(v, item))
}
case reflect.Struct: // ((name value) ...)
for !endList(lex) {
lex.consume('(')
if lex.token != scanner.Ident {
panic(fmt.Sprintf("got token %q, want field name", lex.text()))
}
name := lex.text()
lex.next()
read(lex, v.FieldByName(name))
lex.consume(')')
}
case reflect.Map: // ((key value) ...)
v.Set(reflect.MakeMap(v.Type()))
for !endList(lex) {
lex.consume('(')
key := reflect.New(v.Type().Key()).Elem()
read(lex, key)
value := reflect.New(v.Type().Elem()).Elem()
read(lex, value)
v.SetMapIndex(key, value)
lex.consume(')')
}
default:
panic(fmt.Sprintf("cannot decode list into %v", v.Type()))
}
}
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 12.6. EXAMPLE: DECODING S-EXPRESSIONS 347
func endList(lex *lexer) bool {
switch lex.token {
case scanner.EOF:
panic("end of file")
case ')':
return true
}
return false
}
Final ly, wewrap upthe parserinanexp orted function Unmarshal,shown below,thathides
some ofthe rough edges ofthe imp lementation.Error s encountere d during parsingresultina
panic, so Unmarshal us esadefer red cal l to recov erfro m thepanic (§5.10) andretur n an erro r
mess age ins tead.
// Unmarshal parses S-expression data and populates the variable
// whose address is in the non-nil pointer out.
func Unmarshal(data []byte, out interface{}) (err error) {
lex := &lexer{scan: scanner.Scanner{Mode: scanner.GoTokens}}
lex.scan.Init(bytes.NewReader(data))
lex.next() // get the first token
defer func() {
// NOTE: this is not an example of ideal error handling.
if x := recover(); x != nil {
err = fmt.Errorf("error at %s: %v", lex.scan.Position, x)
}
}()
read(lex, reflect.ValueOf(out).Elem())
return nil
}
Apro duc tion-quality imp lementation shouldnever panic for any inp utand shouldrep ort an
infor mat ive error for every mishap, perhaps wit h alinenumberoroffset. Non etheless, we
hope thisexampleconve yssom e ide a of whatshappeningunder the hood ofthe packages like
encoding/json,and howyou can use reflec tion topopu latedat a st ruc tures.
Exercis e 12.8: The sexpr.Unmarshal func tion, like json.Marshal,requires the complete
inputinabyteslice beforeitcan beg in de coding. Define a sexpr.Decoder type that, like
json.Decoder,allowsasequence ofvalues tobedecoded fro m an io.Reader.Change
sexpr.Unmarshal to use thisnew typ e.
Exercis e 12.9: Wr ite a token-b ased API for decodingS-expressions,fol low ing the sty leof
xml.Decoder (§7.14). You will need ve typ es of tokens: Symbol, String, Int, StartList,
and EndList.
Exercis e 12.10: Extend sexpr.Unmarshal to handlethe boole ans, floating-p ointnumbers,
andint erfaces enco dedbyyoursolut ion toExercis e 12.3. (Hint: todecodeint erfaces, you will
ne e d amapping fro m thenameofeachsup por ted typ e to its reflect.Type.)
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
348 CHAPTER 12. REFLECTION
12.7. Accessing Struct Field Tags
In Sec tion 4.5 weusedstr uct el d tags to modif y theJSONencodingofGostr uct values. The
json eldtag lets uscho ose alt ernat ive eldnames andsup pressthe out put of emp tyfields.In
this sec tion,well see how toaccess eldtags usingreflec tion.
In a web ser ver,the rs t thingmostHTTPhandler functions doisext ractthe requestparame-
ters int o lo cal variables. Well define a utilit y func tion, params.Unpack,thatusesstr uct eld
tags tomakewriting HTTPhandlers(§7.7) moreconvenient.
First,well showhow itsused. The search func tionbelow isanHTTPhandler.Itdefinesa
var iable cal le d data of ananony mou s st ruc t type whose elds cor respond tothe HTTP
re questparameters. Thestr uctsfieldtags specif y theparameter names, whichare often short
andcrypt icsince space ispre cious inaURL. The Unpack func tionpopu lates the str uct fro m
therequestsothatthe parameterscan beaccessedconvenientlyand wit h an appropriatetyp e.
gopl.io/ch12/search
import "gopl.io/ch12/params"
// search implements the /search URL endpoint.
func search(resp http.ResponseWriter, req *http.Request) {
var data struct {
Labels []string `http:"l"`
MaxResults int `http:"max"`
Exact bool `http:"x"`
}
data.MaxResults = 10 // set default
if err := params.Unpack(req, &data); err != nil {
http.Error(resp, err.Error(), http.StatusBadRequest) // 400
return
}
// ...rest of handler...
fmt.Fprintf(resp, "Search: %+v\n", data)
}
The Unpack func tionbelow doesthree things. First,itcal ls req.ParseForm() to parse the
re quest. Thereafter, req.Form cont ainsall the parameters, regardlessofwhether the HTTP
clientusedthe GET orthe POSTrequestmet hod.
Next, Unpack builds a mappingfro m the effec tiv e name ofeach eldtothe var iable for that
eld. The effec tive namemay differfro m theactualnameifthe eldhas a tag . The Field
method of reflect.Type returnsareflect.StructField that provides infor mat ionabout
thetyp e of each eldsuchasits name, typ e,and opt ion altag . The Tag eldisa
reflect.StructTag,whichisastr ing typ e that provides a Get method toparse and ext ract
thesubst ringfor a par tic ularkey,suchas http:"..." in thiscas e.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 12.7. ACCESSING STRUCT FIELD TAGS 349
gopl.io/ch12/params
// Unpack populates the fields of the struct pointed to by ptr
// from the HTTP request parameters in req.
func Unpack(req *http.Request, ptr interface{}) error {
if err := req.ParseForm(); err != nil {
return err
}
// Build map of fields keyed by effective name.
fields := make(map[string]reflect.Value)
v:=reflect.ValueOf(ptr).Elem() // the struct variable
for i := 0; i < v.NumField(); i++ {
fieldInfo := v.Type().Field(i) // a reflect.StructField
tag := fieldInfo.Tag // a reflect.StructTag
name := tag.Get("http")
if name == "" {
name = strings.ToLower(fieldInfo.Name)
}
fields[name] = v.Field(i)
}
// Update struct field for each parameter in the request.
for name, values := range req.Form {
f:=fields[name]
if !f.IsValid() {
continue // ignore unrecognized HTTP parameters
}
for _, value := range values {
if f.Kind() == reflect.Slice {
elem := reflect.New(f.Type().Elem()).Elem()
if err := populate(elem, value); err != nil {
return fmt.Errorf("%s: %v", name, err)
}
f.Set(reflect.Append(f, elem))
}else {
if err := populate(f, value); err != nil {
return fmt.Errorf("%s: %v", name, err)
}
}
}
}
return nil
}
Final ly, Unpack it erates overthe name/value pairsofthe HTTPparametersand updates the
correspondingstr uct elds.Recal l that the sameparameter namemay appear morethan
on ce. Ifthishappens,and the eldisaslice,thenall the values ofthatparameter areacc umu-
late d into the slice.Other wis e,the eldisrep eatedlyoverwritt ensothatonlythe lastvalue
hasany effec t.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
350 CHAPTER 12. REFLECTION
The populate func tiontakes careofsetting a singlefield v (orasingleelementofaslice eld)
from a parameter value.For now,itsup por tsonlystr ings, signe d integers,and boole ans.
Su pportingother typ es is lef t as an exercise.
func populate(v reflect.Value, value string) error {
switch v.Kind() {
case reflect.String:
v.SetString(value)
case reflect.Int:
i, err := strconv.ParseInt(value, 10, 64)
if err != nil {
return err
}
v.SetInt(i)
case reflect.Bool:
b, err := strconv.ParseBool(value)
if err != nil {
return err
}
v.SetBool(b)
default:
return fmt.Errorf("unsupported kind %s", v.Type())
}
return nil
}
If weadd the server hand ler toaweb ser ver,thismig htbeatypic al session:
$gobuild gopl.io/ch12/search
$./search &
$./fetch 'http://localhost:12345/search'
Search: {Labels:[] MaxResults:10 Exact:false}
$./fetch 'http://localhost:12345/search?l=golang&l=programming'
Search: {Labels:[golang programming] MaxResults:10 Exact:false}
$./fetch 'http://localhost:12345/search?l=golang&l=programming&max=100'
Search: {Labels:[golang programming] MaxResults:100 Exact:false}
$./fetch 'http://localhost:12345/search?x=true&l=golang&l=programming'
Search: {Labels:[golang programming] MaxResults:10 Exact:true}
$./fetch 'http://localhost:12345/search?q=hello&x=123'
x: strconv.ParseBool: parsing "123": invalid syntax
$./fetch 'http://localhost:12345/search?q=hello&max=lots'
max: strconv.ParseInt: parsing "lots": invalid syntax
Exercis e 12.11: Wr ite the cor responding Pack func tion. Given a str uct value, Pack should
return a URL incorporat ingthe parameter values fro m thestr uct.
Exercis e 12.12: Extend the eldtag not ation toexpress parameter validity requirements. For
example, a str ing mig htneed tobeavalid emai l addressorcre dit-c ardnumber, and anint eger
mig htneed tobeavalid US ZIP code.Modif y Unpack to checkthese requirements.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 12.8. DISPLAYING THE METHODS OF A TYPE 351
Exercis e 12.13: Mo dif y theS-expressionencoder (§12.4) anddecoder (§12.6) sothatthe y
honorthe sexpr:"..." eldtag in a similarmanner to encoding/json (§4.5).
12.8. Displaying the Methods ofaType
Ourfinalexampleofreflec tion uses reflect.Type to print the typ e of anarbit rar y value and
enumerateits met hods:
gopl.io/ch12/methods
// Print prints the method set of the value x.
func Print(x interface{}) {
v:=reflect.ValueOf(x)
t:=v.Type()
fmt.Printf("type %s\n", t)
for i := 0; i < v.NumMethod(); i++ {
methType := v.Method(i).Type()
fmt.Printf("func (%s) %s%s\n", t, t.Method(i).Name,
strings.TrimPrefix(methType.String(), "func"))
}
}
Both reflect.Type and reflect.Value have a met hod cal le d Method.Each t.Method(i)
call retur nsanins tance of reflect.Method,astr uct typ e that descr ibesthe nameand typ e of
asinglemet hod.Each v.Method(i) call retur nsareflect.Value repres enting a met hod
value (§6.4), thatis, a met hod boundtoits receiver. Usingthe reflect.Value.Call method
(w hichwedonthavespace toshowhere), itspossibletocal l Valuesofkind Func li kethisone,
butthisprogram needsonlyits Type.
Here are the methodsbelon gingtotwo typ es, time.Duration and *strings.Replacer:
methods.Print(time.Hour)
// Output:
// type time.Duration
// func (time.Duration) Hours() float64
// func (time.Duration) Minutes() float64
// func (time.Duration) Nanoseconds() int64
// func (time.Duration) Seconds() float64
// func (time.Duration) String() string
methods.Print(new(strings.Replacer))
// Output:
// type *strings.Replacer
// func (*strings.Replacer) Replace(string) string
// func (*strings.Replacer) WriteString(io.Writer, string) (int, error)
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
352 CHAPTER 12. REFLECTION
12.9. A Word ofCaution
Thereisalot moretothe reflec tion API thanwehavespace toshow, but the pre cedingexam-
ples giveanide a of whatispossible. Reflec tion isapow erfuland expressivetool, but it should
be usedwit h care , forthree reasons.
The rs t re asonisthatreflec tion-b ased codecan befrag i le.For every mistake thatwou ld
caus e acompi ler torep ort a typ e er ror,there isacor respondingway tomisus e reec tion,but
whereasthe compi ler rep ortsthe mistake atbui ld time,areflec tion error isrep orted dur ing
exec ution asapanic, possibly lon g af ter the program was writt enorevenlon g af ter ithas
st arted running.
If the readList func tion(§12.6), for example, shouldreadastr ing fro m theinp utwhi le
popu lat ingavar iable oftyp e int,the cal l to reflect.Value.SetString wi l l panic. Most
prog ramsthatuse reflec tion havesimi lar hazards,and con siderable careisrequired tokeep
trackofthe typ e,addressabi lit y,and settabi lit y of each reflect.Value.
Thebestway toavoid thisfrag i lity istoens ure thatthe use ofreflec tion isfullyenc apsulated
within yourpackageand,ifpossible, avoid reflect.Value in favor of specifictyp es in your
packagesAPI,torestr ict inp uts tolegal values. Ifthisisnot possible, per for m addition al
dy namic che cks beforeeachrisky operat ion. Asanexamplefro m thestand ard librar y,when
fmt.Printf applies a verb to an inappro priateoperand, itdoesnot panic mysteriou sly but
pr ints an infor mat ive error message . Theprogram still has a bug , butitiseasier todiagnos e.
fmt.Printf("%d %s\n", "hello", 42) // "%!d(string=hello) %!s(int=42)"
Reec tion als o re duces the safet y andacc uracyofaut omated refac tor ing and analysistools,
becaus e they cantdeter mineorrelyontyp e infor mat ion.
Thesecon d re asontoavoid reflec tion isthatsince typ es serveasafor m of documentation and
theoperat ions ofreflec tion cannot besubjec t to statictyp e ch eck ing , he avi lyreflec tive codeis
of ten hardtounderst and.Always caref ullydocumentthe exp ected typ es andother invar iants
of functions thatacceptan interface{} or a reflect.Value.
Thethirdreasonisthatreflec tion-b ased functions may beone ortwo ordersofmag nitude
slow erthancodespeci alizedfor a par tic ulartyp e.Inatypic al prog ram, themaj ority offunc-
tion s arenot relevanttothe overal l perfor mance,soitsfine touse reflec tion whenitmakes the
prog ram clearer. Testing isapar tic ularlygood tfor reflec tion since mosttests use small dat a
sets. But for functions onthe criticalpat h, reec tion isbestavoide d.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
13
Lo w-Level Pro gramming
Thedesig n of Goguarante esanumberofsafet y prop erties thatlimitthe ways in whichaGo
prog ram can ‘‘go wro ng.’’ During compi lat ion, typ e ch eck ing detec tsmostatt emp tstoapp l y
an operat iontoavalue thatisinappro priatefor itstyp e,for ins tance,subtrac tingone str ing
from another.Str ict rules for typ e conv ersions pre ventdirec t accesstothe int ernalsofbui lt-in
typeslikestr ings, maps, slices, andchannel s.
Fo r er ror s that cannot bedetec ted statically, suchasout-of-b ounds array accessesornil
pointerdereferences, dynamic che cks ensure thatthe program immediate lyter minates wit h
an infor mat ive error whene ver a forbidden operat ionocc urs. Aut omat ic memory manage-
ment (garb agecol lec tion)eliminates ‘‘us e af ter fre e’’ bugs, as wel l as mostmemor y le aks.
Many imp lementation det ails are inaccessibletoGoprograms. Thereisnoway todis cov er
thememor y layout of anaggregatetyp e li kea str uct,orthe machinecodefor a function, or
theidentity ofthe operat ingsystemthreadonwhichthe cur rentgoroutine isrunning.
In deed,the Gosch edu ler fre ely mov esgoroutinesfro m on e thre adtoanother.Apointeriden-
tifies a variablewit houtrevealingthe var iablesnumer ic address. Addressesmay change asthe
garb agecol lec tor mov esvar iables; point ers are transp arent lyupdated.
Together,these featuresmakeGoprograms, especi ally fai lingones, morepre dic table andless
myster ious thanprogramsinC,the quintess ent ial low-le vel langu age. Byhidingthe under-
ly ing det ails,the y also makeGoprogramshig hly por table,since the langu agesemantics are
large lyindep endentofany par tic ularcompi ler,operat ingsystem, orCPU archit ecture. (No t
entire lyindep endent: som e det ails leakthrough,suchasthe wordsize ofthe pro cessor, the
order ofevaluation of cer tain expressions,and the set ofimp lementation restr ictions imp osed
by the compi ler.)
Occasionally, wemay cho ose tofor feitsom e of these helpf ulguarante estoachie vethe hig hest
possible per for mance,toint ero peratewit h librar ies wr itt eninother langu ages, ortoimp le-
ment a functionthatcannot beexpress ed in pureGo.
353
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
354 CHAPTER 13. LOW-LEVEL PROGRAMMING
In thischapt er, well see how the unsafe packagelets usstepoutside the usu alrules, andhow
to use the cgo to oltocre ate Gobindings for C librar ies andoperat ingsystemcal ls.
Theappro ach esdes crib edinthischapt ershouldnot beusedfrivo lou sly.Wit houtcaref ul
attent ion todet ail, the y maycause the kinds ofunp redic table,ins crutable, non-lo cal failures
with whichCprogrammersare unhappi lyacquainte d. Us e of unsafe also voids Goswar ranty
of compatibi lit y with future rele ases, since,whether intended orinadver tent, itiseasy to
dep endonuns pecified imp lementation det ails thatmay change unexp ectedly.
The unsafe packageisrat her mag ical.Alt hough itapp earstobearegu lar packageand is
imported inthe usu alway,itisactuallyimp lemente d by the compi ler.Itprovides accesstoa
numb erofbui lt-in langu agefeaturesthatare not ordinar ily avai lable because the y exp ose
det ails ofGosmemor y layout.Present ing these featuresasasep aratepackagemakes the rare
occasions onwhichthe y areneeded morecon spicuou s.Als o,som e enviro nmentsmay restr ict
theuse ofthe unsafe packagefor sec urity reasons.
Package unsafe is used extensive lywit hin low-le vel packages like runtime, os, syscall,and
net that interac t with theoperat ingsystem, but isalmostnever needed byordinar y prog rams.
13.1. unsafe.Sizeof, Alignof,and Offsetof
The unsafe.Sizeof func tionrep ortsthe size in bytes ofthe represent ation of itsoperand,
whichmay beanexpressionofany typ e; theexpressionisnot evaluate d. Acal l to Sizeof is a
cons tantexpressionoftyp e uintptr,sothe resultmay beusedasthe dimensionofanarray
type,ortocompute other cons tants.
import "unsafe"
fmt.Println(unsafe.Sizeof(float64(0))) // "8"
Sizeof reportsonlythe size ofthe xe d part ofeachdat a st ruc ture, likethe point erand lengt h
of a str ing , butnot indirec t partslikethe contentsofthe str ing . Typicalsizes for all non-
ag gregateGotyp es areshown below,thoug h theexac t sizes may varybytoolchain. For
portabilit y,weve given the sizes ofreference typ es (ortyp es cont ainingreferences) in ter msof
word s,where a wordis4bytes ona32-bit platfor m and8bytes ona64-bit platfor m.
Comp uters loadand store values fro m memory mostefficientlywhenthose values are
prop erly alig ned.For example, the addressofavalue ofatwo-bytetyp e such as int16 should
be anevennumber, the addressofafour-byte value suchasarune shouldbeamultipleof
four,and the addressofaneig ht-bytevalue suchasafloat64, uint64,or64-bit point er
shouldbeamultipleofeig ht. Alig nmentrequirements ofhig her multiples areunu sual, even
forlargerdat a typessuchas complex128.
Fo r this reason, the size ofavalue of an aggregatetyp e (a str uct orarray) is at least the sum of
thesizes ofits elds orelements but may begre aterdue tothe presenceof ‘‘holes.’’ Ho les are
unus edspaces addedbythe compi ler toens ure thatthe fol low ing eldorelementispro perly
alig ned rel ative tothe start ofthe str uct orarray.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 13.1. UNSAFE.SIZEOF,ALIGNOF,AND OFFSETOF 355
Ty peSiz e
bool 1byte
int
N
, uint
N
, float
N
, complex
NN
/8bytes (for example, float64 is 8 bytes)
int, uint, uintptr 1word
*T 1word
string 2words (data,len)
[]T 3words (data,len, cap)
map 1word
func 1word
chan 1word
interface 2words (type,value)
Thelangu agespecification doesnot guarante e that the order in which elds are declare d is the
order in whichthe y arelaid out inmemor y,sointhe ory a compi ler isfre e to rearrange them,
although aswewrite this, non e do.Ifthe typ es of a str uctsfields are ofdif ferentsizes, itmay
be morespace-efficienttodeclare the elds inanorder thatpacks themastig htlyaspossible.
Thethree str uctsbelow havethe same elds,but the rs t re quires upto50% morememor y
than the other two:
// 64-bit 32-bit
struct{ bool; float64; int16 } // 3 words 4words
struct{ float64; int16; bool } // 2 words 3words
struct{ bool; int16; float64 } // 2 words 3words
Thedet ails ofthe alig nmentalgor it hmare beyon d thescope ofthisbook, anditscer tain lynot
worthwor rying about every str uct,but efficientpacking may makefre quentlyallocated dat a
st ruc tures morecompact and therefore faster.
The unsafe.Alignof func tionrep ortsthe required alig nmentofits argumentstyp e.Like
Sizeof,itmay beapp lie d to anexpressionofany typ e,and ityieldsacon stant.Typic ally,
boole an andnumer ic typesare alig ned totheir size (uptoamaximum of8bytes) andall other
typesare word-alig ned.
The unsafe.Offsetof func tion, whose operandmustbeaeldselec tor x.f,computesthe
of fsetoffield f re lat ive tothe start ofits enclosingstr uct x,accounting for holes, if any.
Figure13.1 shows a str uct var iable x andits memor y layout on typic al 32- and64-bit Go
implementation s.The gray reg ions are holes.
var x struct {
abool
bint16
c[]int
}
Thetable below shows the results ofapp l yingthe three unsafe func tions to x itself andtoeach
of itsthree elds:
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
356 CHAPTER 13. LOW-LEVEL PROGRAMMING
Figure 13.1. Ho les in a str uct.
Typical 32-bit platform:
Sizeof(x) =16 Alignof(x) =4
Sizeof(x.a) = 1 Alignof(x.a) = 1 Offsetof(x.a) = 0
Sizeof(x.b) = 2 Alignof(x.b) = 2 Offsetof(x.b) = 2
Sizeof(x.c) = 12 Alignof(x.c) = 4 Offsetof(x.c) = 4
Typical 64-bit platform:
Sizeof(x) = 32 Alignof(x) =8
Sizeof(x.a) = 1 Alignof(x.a) = 1 Offsetof(x.a) = 0
Sizeof(x.b) = 2 Alignof(x.b) = 2 Offsetof(x.b) = 2
Sizeof(x.c) = 24 Alignof(x.c) = 8 Offsetof(x.c) = 8
Despit e their names, these functions are not infac t unsafe , andthe y maybehelpf ulfor under-
st andingthe layoutofraw memor y in a program whenopt imizingfor space.
13.2. unsafe.Pointer
Most point ertyp es arewritt en *T,meaning ‘‘apoint ertoavar iable oftyp e T.’’ The
unsafe.Pointer type isaspeci al kind ofpoint erthatcan holdthe addressofany var iable.Of
course,wecantindirec t thro ugh an unsafe.Pointer using *p becaus e we dontknowwhat
type thatexpressionshouldhave. Likeordinar y pointers, unsafe.Pointersare comparable
andmay becompare d with nil,whichisthe zerovalue ofthe typ e.
An ordinar y *T pointermay beconverted toan unsafe.Pointer,and an unsafe.Pointer
maybeconverted back to anordinar y pointer, not necessarily ofthe sametyp e *T.Bycon-
vertinga*float64 pointertoa*uint64,for ins tance,wecan insp ect the bit pattern ofaoat-
ing-p ointvar iable:
package math
func Float64bits(f float64) uint64 { return *(*uint64)(unsafe.Pointer(&f)) }
fmt.Printf("%#016x\n", Float64bits(1.0)) // "0x3ff0000000000000"
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 13.2. UNSAFE.POINTER 357
Through the resulting point er, wecan updatethe bit pattern too.Thisishar mlessfor a oat-
ing-p ointvar iable since anybit pattern islegal,but ingeneral, unsafe.Pointer conv ersions
let uswrite arbit rar y values tomemor y andthu s su bvert the typ e system.
An unsafe.Pointer mayals o be converted toauintptr that holds the point ersnumer ic
value,letting usper for m ar ithmeticonaddresses. (Recal l from Chapt er3 thatauintptr is an
unsig ned int egerwide enough torepresent anaddress.) Thisconversiontoo may beapp lie d
in reverse,but again, conv ertingfro m a uintptr to an unsafe.Pointer maysubvert the typ e
systemsince not all numbers are valid addresses.
Many unsafe.Pointer values arethu s intermediaries for convertingordinar y pointers to raw
numericaddressesand backagain. Theexamplebelow takes the addressofvar iable x,adds
theoffsetofits b eld, convertsthe resulting addressto *int16,and through thatpoint er
up dates x.b:
gopl.io/ch13/unsafeptr
var x struct {
abool
bint16
c[]int
}
// equivalent to pb := &x.b
pb := (*int16)(unsafe.Pointer(
uintptr(unsafe.Pointer(&x)) + unsafe.Offsetof(x.b)))
*pb = 42
fmt.Println(x.b) // "42"
Although the syntaxiscumbers omeperh aps nobad thingsince these featuresshouldbe
us edsparing lydo not betempt edtoint roduce temporar y var iables oftyp e uintptr to bre ak
thelines. Thiscodeisincor rec t:
// NOTE: subtly incorrect!
tmp := uintptr(unsafe.Pointer(&x)) + unsafe.Offsetof(x.b)
pb := (*int16)(unsafe.Pointer(tmp))
*pb = 42
Thereasonisver y su btle. Som e garb agecol lec tor s move var iables around inmemor y to
re duce frag mentation or bookkeeping . Garb agecol lec tor s of thiskindare known as moving
GCs.Whenavar iable ismov ed, all point ers thatholdthe addressofthe old location mustbe
up dated topoint tothe newone.Fro m thepersp ectiveofthe garb agecol lec tor,an
unsafe.Pointer is a point erand thu s itsvalue mustchange asthe var iable mov es, but a
uintptr is justa numbersoits value mustnot change . Theincor rec t co de ab ove hi d esa
pointer from the garb agecol lec tor inthe non-p ointervar iable tmp.Bythe timethe secon d
st atement exe cut es, the var iable x couldhavemov edand the numberin tmp wouldnolon g er
be the address &x.b.The thirdstatement clobb ers anarbit rar y memory locat ionwit h the
value 42.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
358 CHAPTER 13. LOW-LEVEL PROGRAMMING
Thereare myr iadpat holog ical variation s on thistheme.After thisstatement has exec ute d:
pT := uintptr(unsafe.Pointer(new(T))) // NOTE: wrong!
thereare nopoint ers thatrefer tothe var iable create d by new,sothe garb agecol lec tor is
entitled torec ycleits storagewhenthisstatement completes, afterwhich pT cont ainsthe
addresswhere the var iable was but isnolon g er.
No cur rentGoimp lementation usesamov ing garb agecol lec tor (though fut ure imp lemen-
tation s mig ht), but thisisnoreasonfor compl acency : currentversions of Go do mov e some
var iables around inmemor y.Recal l from Sec tion 5.2 thatgoroutine stacksgrow as needed.
Wh enthishappens,all var iables onthe old stack may berelocated toanew,largerstack,sowe
cannot relyonthe numer ic value ofavar iablesaddressremainingunchange d thro ughoutits
lifet ime.
At the timeofwriting , thereislit tle cle ar guid ance onwhatGoprogrammersmay relyupon
af ter an unsafe.Pointer to uintptr conv ersion(seeGoissue7192), sowestron gly recom-
mend thatyou assume the bareminimum. Tre atall uintptr values as if the y cont ain the
for mer addressofavar iable,and minimize the numberofoperat ions bet weenconvertingan
unsafe.Pointer to a uintptr andusingthat uintptr.Inour rs t exampleabove , thethree
op erat ionsconv ersiontoauintptr,addition of the eldoffset, conv ersionbackal l
appeared wit hin a singleexpression.
Wh encal lingalibrar y func tionthatretur nsauintptr,such as those below fro m the reflect
package, the resultshouldbeimmediate lyconverted toan unsafe.Pointer to ens ure thatit
cont inues topoint tothe samevar iable.
package reflect
func (Value) Pointer() uintptr
func (Value) UnsafeAddr() uintptr
func (Value) InterfaceData() [2]uintptr // (index 1)
13.3. Example: Deep Equivalence
The DeepEqual func tionfro m the reflect packagerep ortswhether two values are ‘‘de eply’’
equal. DeepEqual comp aresbasic values as if bythe bui lt-in == op erator ; forcomposite val-
ues, ittraversesthemrec ursively, comparing cor respondingelements. Because itwor ksfor
anypair ofvalues, evenonesthatare not comparable wit h ==,itfind s widespreaduse intests.
Thefol low ing testuses DeepEqual to compare two []string values:
func TestSplit(t *testing.T) {
got := strings.Split("a:b:c", ":")
want := []string{"a", "b", "c"};
if !reflect.DeepEqual(got, want) { /* ... */ }
}
Although DeepEqual is convenient, its dist inc tions can seemarbit rar y.For example, itdoesnt
consider a nilmap equ altoanon-ni l empt y map, nor a nil slice equ altoanon-ni l empt y on e:
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 13.3. EXAMPLE: DEEP EQUIVALENCE 359
var a, b []string = nil, []string{}
fmt.Println(reflect.DeepEqual(a, b)) // "false"
var c, d map[string]int = nil, make(map[string]int)
fmt.Println(reflect.DeepEqual(c, d)) // "false"
In thissec tion well define a function Equal that comp aresarbit rar y values. Like DeepEqual,
it comparesslices andmaps bas edontheir elements, but unlike DeepEqual,itcon sidersanil
slice (ormap) equ altoanon-ni l empt y on e.The basic rec ursionoverthe arguments can be
done wit h reec tion,usingasimi lar approach to the Display prog ram we saw in Sec tion 12.3.
As usu al, wedefine anunexp orted function, equal,for the rec ursion. Dontwor ryabout the
seen parameter justyet. For eachpair ofvalues x and y to becompare d, equal ch ecksthat
both (or neither) arevalid andche cks thatthe y have the sametyp e.The resultofthe function
is define d as a set ofswitch cas es that comp are two values ofthe sametyp e.For reasons of
sp ace,weve omitt edseveral cas es since the pattern shouldbefami liar bynow.
gopl.io/ch13/equal
func equal(x, y reflect.Value, seen map[comparison]bool) bool {
if !x.IsValid() || !y.IsValid() {
return x.IsValid() == y.IsValid()
}
if x.Type() != y.Type() {
return false
}
// ...cycle check omitted (shown later)...
switch x.Kind() {
case reflect.Bool:
return x.Bool() == y.Bool()
case reflect.String:
return x.String() == y.String()
// ...numeric cases omitted for brevity...
case reflect.Chan, reflect.UnsafePointer, reflect.Func:
return x.Pointer() == y.Pointer()
case reflect.Ptr, reflect.Interface:
return equal(x.Elem(), y.Elem(), seen)
case reflect.Array, reflect.Slice:
if x.Len() != y.Len() {
return false
}
for i := 0; i < x.Len(); i++ {
if !equal(x.Index(i), y.Index(i), seen) {
return false
}
}
return true
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
360 CHAPTER 13. LOW-LEVEL PROGRAMMING
// ...struct and map cases omitted for brevity...
}
panic("unreachable")
}
As usu al, wedontexp osethe use ofreflec tion inthe API,sothe exp orted function Equal
mu stcal l reflect.ValueOf on itsarguments:
// Equal reports whether x and y are deeply equal.
func Equal(x, y interface{}) bool {
seen := make(map[comparison]bool)
return equal(reflect.ValueOf(x), reflect.ValueOf(y), seen)
}
type comparison struct {
x, y unsafe.Pointer
treflect.Type
}
To ens ure thatthe algor it hmter minates evenfor cyc lic dat a st ruc tures, itmustrecordwhich
pairsofvar iables ithas already compare d andavoid comp aring themasecon d time. Equal
al locatesaset of comparison st ruc ts, each holdingthe addressoftwo var iables (represent edas
unsafe.Pointer values) andthe typ e of the comparison. Weneed torecordthe typ e in addi-
tion tothe addressesbecause dif ferentvar iables can havethe sameaddress. For example, if x
and y arebot h ar rays, x and x[0] have the sameaddress, as do y and y[0],and itisimp ortant
to distinguish whether wehavecompare d x and y or x[0] and y[0].
Once equal hasest ablishe d that its arguments havethe sametyp e,and beforeitexe cut esthe
sw itch,itche cks whether itiscomparing two var iables ithas already seenand,ifso, ter minates
therec ursion.
// cycle check
if x.CanAddr() && y.CanAddr() {
xptr := unsafe.Pointer(x.UnsafeAddr())
yptr := unsafe.Pointer(y.UnsafeAddr())
if xptr == yptr {
return true // identical references
}
c:=comparison{xptr, yptr, x.Type()}
if seen[c] {
return true // already seen
}
seen[c] = true
}
Heresour Equal func tion in action:
fmt.Println(Equal([]int{1, 2, 3}, []int{1, 2, 3})) // "true"
fmt.Println(Equal([]string{"foo"}, []string{"bar"})) // "false"
fmt.Println(Equal([]string(nil), []string{})) // "true"
fmt.Println(Equal(map[string]int(nil), map[string]int{})) // "true"
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 13.4. CALLING C CODE WITH CGO 361
It evenwor ksoncyc lic inputs similartothe one thatcausedthe Display func tionfro m
Section12.3 toget stuck inaloop:
// Circular linked lists a -> b -> a and c -> c.
type link struct {
value string
tail *link
}
a, b, c := &link{value: "a"}, &link{value: "b"}, &link{value: "c"}
a.tail, b.tail, c.tail = b, a, c
fmt.Println(Equal(a, a)) // "true"
fmt.Println(Equal(b, b)) // "true"
fmt.Println(Equal(c, c)) // "true"
fmt.Println(Equal(a, b)) // "false"
fmt.Println(Equal(a, c)) // "false"
Exercis e 13.1: Dene a deepcomparisonfunctionthatcon sidersnumbers (of any typ e) equal
if the y dif fer bylessthanone par t in a billion.
Exercis e 13.2: Wr ite a functionthatrep ortswhether its argumentisacyc lic dat a st ruc ture.
13.4. Calling CCode with cgo
AGoprogram might need touse a hardwaredriverimp lemente d in C, quer y an embedde d
database imp lemente d in C++, oruse some linearalgebra routinesimp lemente d in For tran. C
haslon g been the lingu a franca ofprogramming, somanypackages intended for widespread
us e exp ort a C-comp atibleAPI,regardlessofthe langu ageoftheir implementation.
In thissec tion,well bui ld asimpledat a comp ressionprogram thatuses cgo,atoolthatcre ates
Go bindings for C functions.Suchtools are cal le d foreign-f unc tionint erfaces (FFIs), and cgo is
notthe onlyone for Goprograms. SWIG(swig.org)isanother ; it prov ides morecomplex
fe aturesfor int egrat ingwit h C++ class es, but wewontshowithere.
The compress/... su btree ofthe stand ard librar y prov ides comp ressors and decompressors
forpopu lar comp ressionalgor it hms,includingLZW (usedbythe Unix compress command)
andDEFL ATE (usedbythe GNU gzip command). TheAPIsofthese packages varyslig htly
in detai ls, but the y al l prov ide a wrapperfor an io.Writer that comp ressesthe dat a wr itt ento
it,and a wrapperfor an io.Reader that decomp ressesthe dat a re adfro m it.For example:
package gzip // compress/gzip
func NewWriter(w io.Writer) io.WriteCloser
func NewReader(r io.Reader) (io.ReadCloser, error)
Thebzip2 algor it hm, whichisbas edonthe elegantBur rows-Whe eler transfor m,runsslower
than gzipbut yieldssig nificant lybettercompression. The compress/bzip2 packageprovides
adecompressorfor bzip2, but atthe moment the packageprovides nocompressor. Bui lding
on e from scratch isasubst ant ial under tak ing, but there isawel l-do cumente d andhig h-per-
formance open-s ource C implementation,the libbzip2 packagefro m bzip.org.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
362 CHAPTER 13. LOW-LEVEL PROGRAMMING
If the C librar y were small,wewou ldjustpor t it topureGo, and ifits per for mance werenot
cr iticalfor our pur pos es, we wouldbebetteroff inv oking a C program as a helpersubprocess
usingthe os/exec package. Itswhenyou need touse a complex, per for mance-cr iticallibrar y
with a nar row C API thatitmay makesense towrap itusing cgo.For the restofthischapt er,
well wor k thro ugh anexample.
Fr omthe libbzip2 Cpackage, weneed the bz_stream st ruc t type,whichholds the inp utand
output buf fers, andthree C functions: BZ2_bzCompressInit,whichallocates the streams
buffers; BZ2_bzCompress,whichcompressesdat a from the inp utbuf fer tothe out put buf fer ;
and BZ2_bzCompressEnd,whichreleasesthe buf fers. (Dontwor ryabout the mech anics ofthe
libbzip2 package; the pur pos e of thisexampleistoshowhow the par ts ttoget her.)
Well cal l the BZ2_bzCompressInit and BZ2_bzCompressEnd Cfunctions direc tly fro m Go,
butfor BZ2_bzCompress,well define a wrapperfunctioninC,toshowhow itsdon e.The C
source file below lives alongside the Gocode in our package:
gopl.io/ch13/bzip
/* This file is gopl.io/ch13/bzip/bzip2.c, */
/* a simple wrapper for libbzip2 suitable for cgo. */
#include <bzlib.h>
int bz2compress(bz_stream *s, int action,
char *in, unsigned *inlen, char *out, unsigned *outlen) {
s->next_in = in;
s->avail_in = *inlen;
s->next_out = out;
s->avail_out = *outlen;
int r = BZ2_bzCompress(s, action);
*inlen -= s->avail_in;
*outlen -= s->avail_out;
return r;
}
No w letstur n to the Gocode, the rs t part ofwhichisshown below.The import "C" de clara-
tion isspeci al.There isnopackage C,but thisimp ort causes go build to preprocessthe file
usingthe cgo to olbeforethe Gocompi ler seesit.
// Package bzip provides a writer that uses bzip2 compression (bzip.org).
package bzip
/*
#cgo CFLAGS: -I/usr/include
#cgo LDFLAGS: -L/usr/lib -lbz2
#include <bzlib.h>
int bz2compress(bz_stream *s, int action,
char *in, unsigned *inlen, char *out, unsigned *outlen);
*/
import "C"
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 13.4. CALLING C CODE WITH CGO 363
import (
"io"
"unsafe"
)
type writer struct {
wio.Writer // underlying output stream
stream *C.bz_stream
outbuf [64 * 1024]byte
}
// NewWriter returns a writer for bzip2-compressed streams.
func NewWriter(out io.Writer) io.WriteCloser {
const (
blockSize =9
verbosity = 0
workFactor = 30
)
w:=&writer{w: out, stream: new(C.bz_stream)}
C.BZ2_bzCompressInit(w.stream, blockSize, verbosity, workFactor)
return w
}
During preprocessing, cgo generates a temporar y packagethatcontainsGodeclarat ions cor-
resp ondingtoall the C functions and typ es us edbythe file,suchas C.bz_stream and
C.BZ2_bzCompressInit.The cgo to oldis cov ers these typ es by inv oking the C compi ler in a
sp eci al way onthe contentsofthe comment thatpre cedes the imp ort declarat ion.
Thecomment may also contain #cgo direc tivesthatspecif y ext ra opt ion s to the C toolchain.
The CFLAGS and LDFLAGS values cont ribut e ext ra arguments tothe compi ler andlin ker com-
mand s so thatthe y canlocatethe bzlib.h he ader file andthe libbz2.a archivelibrar y.The
exampleassumesthatthese are ins tal le d bene ath /usr on yoursystem. You may need toalt er
or deletethese flags for yourins tal lat ion.
NewWriter makesacal l to the C function BZ2_bzCompressInit to initialize the buf fersfor
thestream. The writer type includes anot her buf fer thatwill beusedtodrain the decom-
pressorsout put buf fer.
The Write method,shown below,feedsthe uncompressed data to the compressor, cal lingthe
func tion bz2compress in a loop unt i l al l thedat a hasbeencon sumed.Obs erve thatthe Go
prog ram mayaccessC typ es li ke bz_stream, char,and uint,Cfunctions like bz2compress,
andevenobj e ct-li keCprepro cessormacros suchas BZ_RUN,all through the C.x notation.The
C.uint type isdistinc t from Gos uint type,even if bot h have the samewidth.
func (w *writer) Write(data []byte) (int, error) {
if w.stream == nil {
panic("closed")
}
var total int // uncompressed bytes written
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
364 CHAPTER 13. LOW-LEVEL PROGRAMMING
for len(data) > 0 {
inlen, outlen := C.uint(len(data)), C.uint(cap(w.outbuf))
C.bz2compress(w.stream, C.BZ_RUN,
(*C.char)(unsafe.Pointer(&data[0])), &inlen,
(*C.char)(unsafe.Pointer(&w.outbuf)), &outlen)
total += int(inlen)
data = data[inlen:]
if _, err := w.w.Write(w.outbuf[:outlen]); err != nil {
return total, err
}
}
return total, nil
}
Each iterat ionofthe looppasses bz2compress theaddressand lengt h of the remaining
portionof data,and the addressand cap acity of w.outbuf.The two lengt h var iables are
pass edbytheir addresses, not their values, sothatthe C functioncan updatethemtoindic ate
howmuchuncompresseddat a was cons ume d andhow muchcompresseddat a was pro duce d.
Each chun k of compresseddat a is thenwritt entothe underly ing io.Writer.
The Close method has a similarstr uctureto Write,usingalooptoflushout any remaining
comp resseddat a from the streamsout put buf fer.
// Close flushes the compressed data and closes the stream.
// It does not close the underlying io.Writer.
func (w *writer) Close() error {
if w.stream == nil {
panic("closed")
}
defer func() {
C.BZ2_bzCompressEnd(w.stream)
w.stream = nil
}()
for {
inlen, outlen := C.uint(0), C.uint(cap(w.outbuf))
r:=C.bz2compress(w.stream, C.BZ_FINISH, nil, &inlen,
(*C.char)(unsafe.Pointer(&w.outbuf)), &outlen)
if _, err := w.w.Write(w.outbuf[:outlen]); err != nil {
return err
}
if r == C.BZ_STREAM_END {
return nil
}
}
}
Up oncomplet ion, Close calls C.BZ2_bzCompressEnd to release the streambuf fers, using
defer to ens ure thatthishappens onall retur n paths. Atthispoint the w.stream pointeris
no lon g ersafetodereference.Tobedefensive , we set itto nil,and add explicitnil che cks to
each met hod,sothatthe program panics if the usermistakenlycal lsamet hod after Close.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
SECTION 13.4. CALLING C CODE WITH CGO 365
No t on lyis writer notcon cur rency-s afe, but con cur rentcal lsto Close and Write couldcause
theprogram tocrash in C code. FixingthisisExercis e 13.3.
Theprogram below, bzipper,isabzip2 comp ressorcommand thatusesour new package. It
behaveslikethe bzip2 commandpresent onmanyUnix systems.
gopl.io/ch13/bzipper
// Bzipper reads input, bzip2-compresses it, and writes it out.
package main
import (
"io"
"log"
"os"
"gopl.io/ch13/bzip"
)
func main() {
w:
=bzip.NewWriter(os.Stdout)
if _, err := io.Copy(w, os.Stdin); err != nil {
log.Fatalf("bzipper: %v\n", err)
}
if err := w.Close(); err != nil {
log.Fatalf("bzipper: close: %v\n", err)
}
}
In the sessionbelow,weuse bzipper to compress /usr/share/dict/words,the systemdic-
tion ary,fro m 938,848 bytes to335,405 bytesab out a thirdofits originalsizethen uncom-
pressitwit h thesystem bunzip2 command. The SHA256 hashisthe samebeforeand after,
giving usconfidence thatthe compressoriswor kingcor rec tly.(If you donthave sha256sum
on yoursystem, use yoursolut ion toExercis e 4.2.)
$gobuild gopl.io/ch13/bzipper
$wc-c</usr/share/dict/words
938848
$sha256sum < /usr/share/dict/words
126a4ef38493313edc50b86f90dfdaf7c59ec6c948451eac228f2f3a8ab1a6ed -
$./bzipper < /usr/share/dict/words | wc -c
335405
$./bzipper < /usr/share/dict/words | bunzip2 | sha256sum
126a4ef38493313edc50b86f90dfdaf7c59ec6c948451eac228f2f3a8ab1a6ed -
Weve demon strated lin kingaClibrar y into a Goprogram. Going inthe other direc tion,its
also possibletocompi leaGoprogram as a staticarc hivethatcan belin ked int o aCprogram
or asashare d librar y that can bedynamic ally loade d by a C program. Weve onlyscratch ed
thesur face of cgo here , andthere ismuchmoretosay aboutmemor y management,point ers,
callbacks, signalhandling, str ings, errno,finalizers, andthe rel ation shipbet weengoroutines
andoperat ingsystemthreads,muchofitver y su btle. Inpar tic ular, the rules for cor rec tly
passingpoint ers fro m Go toCorvice versa are complex, for reasons simi lar tothose we
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
366 CHAPTER 13. LOW-LEVEL PROGRAMMING
discussedinSec tion 13.2, andnot yet author it ative lyspecified.For fur therreading, start wit h
https://golang.org/cmd/cgo.
Exercis e 13.3: Us e sync.Mutex to make bzip2.writer safe forcon cur rentuse bymultiple
goro utines.
Exercis e 13.4: DependingonClibrar ies hasits drawbacks. Provide an alternat ive pure-G o
implementation of bzip.NewWriter that usesthe os/exec packagetorun /bin/bzip2 as a
su bprocess.
13.5. Another WordofCaution
We ended the pre vious chapt erwit h awar ningabout the dow nsides ofthe reflec tion int erface.
That warningapp lies wit h even moreforce tothe unsafe packagedes crib ed in thischapt er.
High-le vel langu ages insu lateprogramsand programmersnot onlyfro m thearc ane specifics
of indiv idu alcomputerins tructionsets, but fro m dep endence onirrelevancies like where in
memory a var iable lives, how big a dat a type is, the det ails ofstr ucturelayout, andahostof
ot her implementation det ails.Because ofthatins ulating layer,itspossibletowrite programs
that aresafeand robustand thatwill run onany operat ingsystemwit houtchange .
The unsafe packagelets programmersreach through the ins ulation touse some cruci al but
ot her wis e inaccessiblefeature , or perhaps toachie vehig her per for mance.The costisusu ally
to por tabilit y andsafet y,soone uses unsafe at onesper il. Our advice onhow and whento
us e unsafe paral lels Knu thscommentsonpremature opt imizat ion, whichwequote d in
Section11.5. Mostprogrammerswill never need touse unsafe at all. Never theless, there will
occasionallybesit uat ions where som e cr iticalpie ce of codecan bebestwritt enusing unsafe.
If caref ulstudy and measurementindic ates that unsafe re allyisthe bestappro ach,restr ict it
to assmall a reg ion as possible, sothatmostofthe program isobliv iou s to its use.
Fo r now, put the lasttwo chapt ers inthe backofyourmind. Write som e su bst ant ial Go
prog rams. Avoid reflect and unsafe;com e back to these chapt ers only if you must.
Me anw hile, happy Goprogramming. Wehop e youenj oywriting Go as much as we do.
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
Index
!,negat ionoperator 63
%,remainder operator 52, 166
&&,short-circuit AND operator 63
&,address-ofoperator 24, 32, 94,
158, 167
&,imp licit158, 167
&^,AND-NOT operator 53
&^,bit-cle ar op erator 53
' quot e ch arac ter 56
*,indirec tion operator 24, 32
++,increment statement 5,37, 94
+,str ing con catenat ionoperator 5,
65
+,unary operator 53
+=, -=,etc., assig nmentoperator 5
-,unary operator 53
--,decrement statement 5,37
... argument139, 142
... ar ray lengt h 82
... parameter 91, 142, 143, 172
... path 292, 299
/*...*/ comment5,25
// comment5,25
:= short var iable decl arat ion5,31,
49
<<,lef t shif t op erator 54
==,comparisonoperator 40, 63
>>,rig htshif t op erator 54
^,bit w ise complementoperator 53
^,exc lusiveORoperator 53
_,blank identifier 7, 38, 95, 120, 126,
287
` backquot e ch arac ter 66
| in templ ate 113
|,bit w ise ORoperator 166, 167
||,short-circuit ORoperator 63
Abst rac t Sy ntaxNot ation One
(ASN.1) 107
abstrac t type 24, 171
abstrac tion, premature 216, 316, 317
ad hoc poly mor phism 211
addressoflocal variable32, 36
addressofstr uct lit eral 103
addressableexpression159, 341
addressablevalue 32
address-ofoperator & 24, 32, 94,
158, 167
ag gregatetyp e 81, 99
Alef programminglangu agexiii
algor it hm
breadt h-firs t search 139, 239
depth-firs t search 136
Fibonacci 37, 218
GCD 37
insertionsor t 101
Liss ajous 15
slice rot ation 86
topolog ical sor t 136
ali asing, point er33
alig nment354
al location
he ap36
memory 36, 71, 89, 169, 209, 322
st ack 36
anch orelement, HTML 122
AND operator &&,short-circuit 63
AND-NOT operator &^ 53
animat ion, GIF 13
anonymou s
func tion22, 135, 236
func tion, defer 146
func tion, rec ursive137
st ruc t eld104, 105, 106, 162
API
encoding 213, 340
er ror 127, 152
package284, 296, 311, 333, 352
runt ime 324
SQL211
systemcal l 196
template 115
token-b ased decoder 213, 215,
347
APLprogramminglangu agexiii
append built-in function88, 90, 91
appendInt example88
argument
... 139, 142
command-line4,18, 33, 43, 179,
180, 290, 313
func tion119
pointer33, 83
slice 86
ar ithmeticexpressionevaluator197
ar ray
comp arison83
lengt h, ... 82
literal 82, 84
type 81
underly ing 84, 88, 91, 187
zerovalue 82
ASCII 56, 64, 66, 67, 305
ASN.1 (Abstrac t Sy ntaxNot ation
One) 107
assemb l y line, cake234
assertion
func tion316
interface typ e 208, 210
367
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
368 INDEX
test 306
type 205, 211
assig nabilit y 38, 175
assig nabilit y,int erface 175
assig nment
implicit38
mu ltiple-value 37
op erator +=, -=,etc.5
op erator s 36, 52
st atement 5, 7, 36, 52, 94, 173
tuple31, 37
asso ciativity,operator 52
atomic operat ion264
attack,HTMLinj e ction 115
attack,SQL inj e ction 211
autoescape example117
back-do or, package315
back-off,exp onent ial 130
backquot e ch arac ter, ` 66
bank examplepackage258, 261, 263
bare return 126
basename example72
behavior,undefine d 260
Benchmark func tion302, 321
bidirec tion altounidirec tion al
ch annel conversion231
binary
op erator s,table of52
semaph ore 262
tree 102
bit vec tor 165
bit-cle ar op erator &^ 53
bit-s et data typ e 77
bit w ise
comp lementoperator ^ 53
op erator s,table of53
OR operator | 166, 167
bl ack-b oxtest310
bl ank identifier _ 7, 38, 95, 120, 126,
287
bl ank imp ort 287
block
file 46
lexic al 46, 120, 135, 141, 212
lo cal 46
package46
univers e 46
blocking profile 324
Blog , Go xvi,326
boiling example29
bool type 63
boole an
cons tant, false 63
cons tant, true 63
zerovalue 30
breadthFirst func tion139
breadt h-firs t search algor it hm139,
239
break st atement 24, 46
break st atement,lab ele d 249
br ittletest317
broadc ast251, 254, 276
Brooks, Fre d xiv
btoi func tion64
buffered channel 226, 231
bufio package9
bufio.NewReader func tion98
bufio.NewScanner func tion9
(*bufio.Reader).ReadRune
method 98
bufio.Scanner type 9
(*bufio.Scanner).Err method 97
(*bufio.Scanner).Scan method 9
(*bufio.Scanner).Split method
99
bufio.ScanWords func tion99
+build comments 296
buildcon straints 296
buildtags 296
buildingpackages 293
built-in function
append 88, 90, 91
cap 84, 232
close 226, 228, 251
complex 61
copy 89
delete 94
imag 61
len 4, 54, 64, 65, 81, 84, 233
make 9, 18, 88, 94, 225
new 34
panic 148, 149
real 61
recover 152
built-in interface, error 196
built-in typ e, error 11, 128, 149,
196
byte slice tostr ing conversion73
byte type 52
ByteCounter example173
bytes package71, 73
bytes.Buffer type 74, 169, 172, 185
(*bytes.Buffer).Grow method
169
(*bytes.Buffer).WriteByte
method 74
(*bytes.Buffer).WriteRune
method 74
(*bytes.Buffer).WriteString
method 74
bytes.Equal func tion86
bzip Ccode362
bzip examplepackage363
bzipper example365
C++ programminglangu agexiv,xv,
361
Cprogramminglangu agexii,xv, 1,
6, 52, 260, 361
cach e, conc urrentnon-blo cking 272
cach e, non-blo cking 275
cake ass emb l y line234
call
by reference 83
by value 83, 120, 158
interface met hod 182
ok value fro m func tion128
callingCfro m Go 361
camel cas e 28
cancel lat ion251, 252
cancel lat ionofHTTPrequest253
cap built-in function84, 232
capacity,channel 226, 232, 233
capacity,slice 88, 89
capt uring iterat ionvar iable 140
capt uring loopvar iable 141, 236,
240
case in typ e switch 212
case, select 245
Celsius type 39
CelsiusFlag func tion181
cf example43
cgo to ol361, 362
<-ch,channel receive 18, 225, 232
ch<-,channel send18, 225, 232
ch aining, met hod 114
chan type 225
ch annel
buffered 226, 231
capacity 226, 232, 233
close 228, 251
closing a 225
communic ation225, 245
comp arison225
conv ersion, bidirec tion alto
unidirec tion al231
draining a 229, 252
make 18, 225
ni l 246, 249
polling246
range ov er229
re ceive <-ch 18, 225, 232
re ceive , non-blo cking 246
re ceive , ok value fro m 229
send ch<- 18, 225, 232
sy nchro nou s 226
type 18
type <-chan T,receive-only230
type chan<- T,send-only230
type,unidirec tion al230, 231
unbuffered 226
zerovalue 225, 246
ch arac ter conv ersion71
ch arac ter test71
charcount example98
chat example254
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
INDEX 369
ch atser ver 253
CheckQuota func tion312, 313
client, emai l 312
client, SMTP 312
clock example220, 222
clockser ver,con cur rent219
close built-in function226, 228,
251
close,channel 228, 251
clos er goro utine 238, 250
closingachannel 225
closure, lexic al 136
cmplx.Sqrt func tion61
co de
format 3, 6, 9, 48
point, Unico de 67
produc tion301
ColoredPoint example161
comma example73
command, test ing a 308
command-lineargument4,18, 33,
43, 179, 180, 290, 313
comment
/*...*/ 5, 25
// 5, 25
do c 42, 296
// Output 326
comments, +build 296
communic atingsequential pro cesses
(CSP) xiii,217
communic ation, channel 225, 245
comp arabilit y 9, 38, 40, 53, 86, 93,
97, 104
comp arison
ar ray 83
ch annel 225
func tion133
interface 184
map96
op erator == 40, 63
op erator s 40, 93
op erator s,table of53
slice 87
st ring65
st ruc t 104
comp ilation,sep arate284
comp lementoperator ^,bit w ise 53
complex built-in function61
complex type 61
comp osite lit eral 14
comp osite typ e xv,14, 81
comp osition,paral lel224
comp osition,typ e xv,107, 162, 189
compress/bzip2 package361
comp ression361
conceptu alint egr ity xiv
concrete typ e 24, 171, 211, 214
conc urrency 17, 217, 257
excessive 241, 242
safe 275
safety 256, 257, 272, 365
with share d var iables 257
conc urrent
clockser ver 219
direc tor y traversal247
echo ser ver 222
non-blo cking cache 272
we b craw ler 239
confinement,ser ial 262
confinement,var iable 261
consistenc y,sequential 268, 269
const de clarat ion14, 75
cons tant
false boole an 63
generator, iota xiii,77
time.Minute 76
time.Second 164
true boole an 63
types, untyped78
cons tants, pre cisionof78
cons traints, bui ld 296
cont ent ion,lock267, 272
cont ext swit c h 280
continue st atement 24, 46
continue st atement,lab ele d 249
cont rac ts, interfaces as 171
cont rol ow 46
conv ersion
bidirec tion altounidirec tion al
ch annel 231
byte slice tostr ing 73
ch arac ter 71
implicit79
narrow ing 40, 55
numeric79
op erat ion40, 55, 64, 71, 78, 79,
173, 187, 194, 208, 231, 353, 358
rune slice tostr ing 71
rune tostr ing 71
st ring71
st ringtobyteslice 40, 73
st ringtoruneslice 71, 88
unsafe.Pointer 356
copy built-in function89
countdown example244, 245, 246
counting semaphore 241
coverage, statement 318, 320
coverage, test 318
coverage_test example319
CPU profile 324
crawl example240, 242, 243
craw ler,con cur rentweb 239
craw ler,web 119
cr iticalsec tion 263, 270, 275
cross-comp ilation 295
cr yptography 55, 83, 121, 325
crypto/sha256 package83
customSort example190
cyclic dat a st ruc ture337
cyclic testdep endency 314
data
race 259, 267, 275
st ruc ture, cyc lic 337
st ruc ture, rec ursive101, 102, 107
type,bit-s et 77
database driver, MySQL284
database/sql package211, 288
daysAgo func tion114
deadbeef 55, 80
de adlo ck233, 240, 265
de clarat ion
const 14, 75
func 3, 29, 119
import 3, 28, 42, 284, 285, 362
method 40, 155
package 2, 28, 41, 285
package-le vel 28
scop e 45, 137
sh adowing 46, 49, 206, 212
short var iable 5, 7, 30, 31
st atement,short var iable 7
struct 99
type 39
var 5, 30
de clarat ions,order of48
de code example, S-expression347
de coder API, token-b ased 213, 215,
347
de coding, S-expression344
de coding, XML 213
de crementstatement -- 5, 37
dedup example97
de ep equivalence 87, 317, 358
default case in select 246
default case in switch 23
default case intyp e switch 212
defer anonymou s func tion146
defer example150, 151
defer st atement 144, 150, 264
defer red functioncal l 144
delete built-in function94
depth-firs t search algor it hm136
dereference,imp licit159
di agram
helloworld su bst ring69
pip eline228
slice cap acity growt h 90
slice ofmonths84
st ringsharing 65
st ruc t hole 355
thumbnail sequence 238
dig italartifac t example178
Dijkstra, Edsger318
Di lbert 100
direc ted acyclic graph136, 284
direc tor y traversal, con cur rent247
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
370 INDEX
discriminated union211, 213, 214
Display func tion333
display example333
display func tion334
displaying met hodsofatyp e 351
Distance func tion156
do c comment42, 296
doc.go do c comment file 42, 296
do cumentation,package296
domain name, imp ort pat h 284
dot . in templ ate 113
down loadingpackages 292
Dr.Strange lov e 336
drainingachannel 229, 252
du example247, 249, 250
dup example9,11, 12
dup lic atesup pression276
dy namic disp atch 183
dy namic typ e,int erface 181
echo example5, 7, 34, 309
echo test 309
echo ser ver,con cur rent222
echo_test.go 310
ef fec tive tests, writing 316, 317
emai l client312
embarrassinglyparal lel235
embedde d st ruc t eld161
embedding, int erface 174
embedding, str uct 104, 161
Employee st ruc t 100
empt y
interface typ e 176
select st atement 245
st ring5, 7, 30
st ruc t 102
encapsulat ion168, 284
encoding API213, 340
enco ding, S-expression338
encoding/json package107
encoding/xml package107, 213
endof file (EOF) 131
enum 77
enviro nment var iable
GOARCH 292, 295
GOMAXPROCS 281, 321
GOOS 292, 295
GOPATH xv i,291, 295
GOROOT 292
equal func tion87, 96
equality,point er32
equivalence,deep87, 317, 358
error built-in interface 196
error built-in typ e 11, 128, 149, 196
er ror API 127, 152
error.Error method 196
errorf func tion143
er ror-handlingstrateg ies 128, 152,
310, 316
errors package196
errors.New func tion196
es cap e
hexade cimal 66
HTML 116
octal66
sequence 10
sequences, table of66
Unico de 68, 107
URL 111
es capingvar iables 36
eval example198
eventmultiplexing244
events 227, 244
Example func tion302, 326
example
autoescape 117
basename 72
boiling 29
ByteCounter 173
bzipper 365
cf 43
charcount 98
chat 254
clock 220, 222
ColoredPoint 161
comma 73
countdown 244, 245, 246
coverage_test 319
crawl 240, 242, 243
customSort 190
dedup 97
defer 150, 151
dig italartifac t 178
display 333
du 247, 249, 250
dup 9, 11, 12
echo 5, 7, 34, 309
eval 198
fetch 16, 148
fetchall 18
findlinks 122, 125, 139
ftoc 29
github 110, 111
graph 99
helloworld 1, 2
http 192, 194, 195
intset 166
issues 112
issueshtml 115
issuesreport 114
jpeg 287
lissajous 14, 22, 35
mandelbrot 62
memo 275, 276, 277, 278, 279
methods 351
movie 108, 110
netcat 221, 223, 227
netflag 78
nonempty 92
outline 123, 133
package, bank 258, 261, 263
package, bzip 363
package, format 332
package, geometry 156
package, http 192
package, links 138
package, memo 273
package, params 348
package, storage 312, 313
package, tempconv 42
package, thumbnail 235
palindrom e 303, 305, 308
params 348
Parse 152
pipeline 228, 230, 231
pl aylist187
rev 86
reverb 223, 224
server 19, 21
sexpr 340
S-expressiondecode347
sha256 83
sleep 179
spinner 218
squares 135
sum 142
surface 59, 203
tempconv 39, 180, 289
temp eratureconversion29
tempflag 181
test of word 303
thumbnail 236, 237, 238
title 153
topoSort 136
trace 146
treesort 102
urlvalues 160
wait 130
word 303, 305, 308
xmlselect 215
appendInt 88
exception 128, 149
excessive con cur rency 241, 242
exclusion, mut ual 262, 267
exclusivelock263, 266, 270
exclusiveORoperator ^ 53
exp onent ial back-off 130
exp ort ofstr uct eld101, 106, 109,
110, 168
export_test.go file 315
Expr.Check method 202
expression
addressable159, 341
evaluator197
method 164
re ceive 225
Expr.Eval method 199
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
INDEX 371
extendingaslice 86
Extensible Marku p Language (XML)
107
exter nal testpackage285, 314
Fahrenheit type 39
fai luremessage , test 306
fallthrough st atement 23, 212
false boole an cons tant63
fetch example16, 148
fetchall example18
fib func tion37, 218
Fibonacci algor it hm37, 218
eld
anonymou s st ruc t 104, 105, 106,
162
embedde d st ruc t 161
exp ort ofstr uct 101, 106, 109,
110, 168
order,str uct 101, 355
sele ctor156
st ruc t 15, 99
tag, omitempty 109
tag, str uct 109, 348
figure
Liss ajous 13
Mandelbro t 63
3-D sur face 58, 203
Fi le Transfer Pro tocol (FTP) 222
file
block46
export_test.go 315
name,Micros oft Windows 72
name,POSIX 72
_test.go 285, 302, 303
findlinks example122, 125, 139
xe d-size stack 124
flag package33, 179
flag
go tool -bench 321
go tool -benchmem 322
go tool -covermode 319
go tool -coverprofile 319
go tool -cpuprofile 324
go tool -nodecount 325
go tool -text 325
go tool -web 326
godoc -analysis 176
go list -f 315
go -race 271
go test -race 274
go test -run 305
go test -v 304
flag.Args func tion34
flag.Bool func tion34
flag.Duration func tion179
flag.Parse func tion34
flag.String func tion34
flag.Value interface 179, 180
floating-p oint
numb er56
precision56, 57, 63, 78
tr unc ation 40, 55
fmt package2
fmt.Errorf func tion129, 196
fmt.Fprintf func tion172
fmt.Printf func tion10
fmt.Println func tion2
fmt.Scanf func tion75
fmt.Sscanf func tion180
fmt.Stringer interface 180, 210
for scop e 47
for st atement 6
forEachNode func tion133
foreig n-functionint erface (FFI) 361
format, code 3, 6, 9, 48
format examplepackage332
formatAtom func tion332
framewor k,web 193
ftoc example29
func de clarat ion3,29, 119
func tion
anonymou s 22, 135, 236
append built-in 88, 90, 91
argument119
assertion316
Benchmark 302, 321
body, missing121
breadthFirst 139
btoi 64
bufio.NewReader 98
bufio.NewScanner 9
bufio.ScanWords 99
bytes.Equal 86
call,defer red 144
call, ok value fro m 128
cap built-in 84, 232
CelsiusFlag 181
CheckQuota 312, 313
close built-in 226, 228, 251
cmplx.Sqrt 61
comp arison133
complex built-in 61
copy built-in 89
daysAgo 114
delete built-in 94
Display 333
display 334
Distance 156
equal 87, 96
errorf 143
errors.New 196
Example 302, 326
fib 37, 218
flag.Args 34
flag.Bool 34
flag.Duration 179
flag.Parse 34
flag.String 34
fmt.Errorf 129, 196
fmt.Fprintf 172
fmt.Printf 10
fmt.Println 2
fmt.Scanf 75
fmt.Sscanf 180
forEachNode 133
formatAtom 332
gcd 37
hand ler 19, 21, 152, 191, 194, 195,
348
html.Parse 121, 125
http.DefaultServeMux 195
http.Error 193
http.Get 16, 18
http.Handle 195
http.HandleFunc 19, 22, 195
http.ListenAndServe 19, 191
http.NewRequest 253
http.ServeMux 193
hypot 120
imag built-in 61
image.Decode 288
image.RegisterFormat 288
incr 33
init 44, 49
intsToString 74
io.Copy 17, 18
ioutil.ReadAll 16, 272
ioutil.ReadDir 247
ioutil.ReadFile 12, 145
io.WriteString 209
itob 64
json.Marshal 108
json.MarshalIndent 108
json.NewDecoder 111
json.NewEncoder 111
json.Unmarshal 110, 114
len built-in 4, 54, 64, 65, 81, 84,
233
links.Extract 138
literal 22, 135, 227
log.Fatalf 49, 130
main 2, 310
make built-in 9, 18, 88, 94, 225
math.Hypot 156
math.Inf 57
math.IsInf 57
math.IsNaN 57
math.NaN 57
mu lti-value d 11, 30, 37, 96, 125,
126
mustCopy 221
net.Dial 220
net.Listen 220
new built-in 34
ni l 132
os.Close 11
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
372 INDEX
os.Exit 16, 34, 48
os.Getwd 48
os.IsExist 207
os.IsNotExist 207
os.IsPermission 207
os.Open 11
os.Stat 247
panic built-in 148, 149
parameter 119
params.Unpack 349
png.Encode 62
PopCount 45
real built-in 61
recover built-in 152
re cursive anony mou s 137
reflect.TypeOf 330
reflect.ValueOf 331, 337
reflect.Zero 345
regexp.Compile 149
regexp.MustCompile 149
resu ltlist119
runtime.Stack 151
SearchIssues 111
sexpr.Marshal 340
sexpr.readList 347
sexpr.Unmarshal 347
sig nature120
sort.Float64s 191
sort.Ints 191
sort.IntsAreSorted 191
sort.Reverse 189
sort.Strings 95, 137, 191
Sprint 330
sqlQuote 211, 212
strconv.Atoi 22, 75
strconv.FormatInt 75
strconv.Itoa 75
strconv.ParseInt 75
strconv.ParseUint 75
strings.Contains 69
strings.HasPrefix 69
strings.HasSuffix 69
strings.Index 289
strings.Join 7, 12
strings.Map 133
strings.NewReader 289
strings.NewReplacer 289
strings.Split 12
strings.ToLower 72
strings.ToUpper 72
template.Must 114
template.New 114
Test 302
time.After 245
time.AfterFunc 164
time.Now 220
time.Parse 220
time.Since 114
time.Tick 244, 246
title 144, 145
type 119, 120
unicode.IsDigit 71
unicode.IsLetter 71
unicode.IsLower 71
unicode.IsSpace 93
unicode.IsUpper 71
unsafe.AlignOf 355
unsafe.Offsetof 355
unsafe.Sizeof 354
url.QueryEscape 111
utf8.DecodeRuneInString 69
utf8.RuneCountInString 69
value 132
var iadic 142, 172
visit 122
WaitForServer 130
walkDir 247
zerovalue 132
garb agecol lec tion xi, xiii, 7, 35, 230,
353, 357
garb agecol lec tor,mov ing 357
GCD algor it hm37
gcd func tion37
geometry examplepackage156
geometry.Point.Distance method
156
gett ermet hod 169
GIF animat ion13
GitHub issuetracker110
github example110, 111
Go
Pl ayg round xvi,326
Blog xvi, 326
issue110, 112, 358
go to ol2,42, 44, 290
go tool -bench flag 321
go tool -benchmem flag 322
go tool -covermode flag 319
go tool -coverprofile flag 319
go tool -cpuprofile flag 324
go tool -nodecount flag 325
go tool pprof 325
go tool -text flag 325
go tool -web flag 326
go tool cover 318, 319
go doc to ol25
go st atement 18, 218
GOARCH enviro nment var iable 292,
295
go build 2, 286, 293, 294
go doc 296
godoc -analysis flag 176
godoc to olxvi,25, 297, 326
go env 292
gofmt to ol3, 4, 44, 286
go get xv i, 2, 292, 293
go help 290
goimports to ol3,44, 286
go install 295
golang.org/x/net/html package
122
golint to ol292
go list 298, 315
go list -f flag 315
GOMAXPROCS enviro nment var iable
281, 321
GOOS enviro nment var iable 292, 295
GOPATH enviro nment var iable xvi,
291, 295
gopl.io repository xvi
go -race flag 271
GOROOT enviro nment var iable 292
goro utine 18, 217, 233, 235
clos er 238, 250
identity 282
le ak 233, 236, 246
monitor261, 277
mu ltiplexing281
vs. OS thread280
go run 2, 294
go test 301, 302, 304
go test -race flag 274
go test -run flag 305
go test -v flag 304
goto st atement 24
graph example99
GraphViz 326
Gr ies emer,Rob ert xi
growth,stack 124, 280, 358
gu ardingmut ex263
half-openint erval 4
hand ler function19, 21, 152, 191,
194, 195, 348
‘‘happ ens before’’ re lat ion226, 257,
261, 277
‘‘hasa’’ re lat ions hip 162
hash table 9, 93
Haskel l prog ramminglangu agexiv
he ap
al location 36
profile 324
var iable 36
helloworld example1,2
helloworld su bst ringdiagram 69
hexade cimal escape 66
hexade cimal literal 55
hidden point er357
Ho are , To nyxiii
hole,str uct 354
HTML
anch orelement122
es cap e 116
injec tion att ack 115
metach arac ter 116
pars er121
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
INDEX 373
html.Parse func tion121, 125
html/template package113, 115
HT TP
GETrequest21, 127, 272, 348
POSTrequest348
re quest, cancellat ionof253
re questmultiplexer 193
http example192, 194, 195
http examplepackage192
(*http.Client).Do method 253
http.DefaultClient var iable 253
http.DefaultServeMux func tion
195
http.Error func tion193
http.Get func tion16, 18
http.Handle func tion195
http.HandleFunc func tion19, 22,
195
http.Handler interface 191, 193
http.HandlerFunc type 194, 203
http.ListenAndServe func tion19,
191
http.NewRequest func tion253
http.Request type 21, 253
(*http.Request).ParseForm
method 22, 348
http.ResponseWriter type 19, 22,
191, 193
http.ServeMux func tion193
hypot func tion120
identifier _,blank 7,38, 95, 120, 126,
287
identifier,qualified 41, 43
identity,goroutine 282
IEEE 754 stand ard 56, 57
if,initializat ionstatement in22,
206
if-else scop e 47
if-else st atement 9,22, 47
imag built-in function61
imagemanipu lat ion121
image package62, 287
image/color package14
image.Decode func tion288
image/png package288
image.RegisterFormat func tion
288
imag inar y literal 61
immut abi lit y 261
immut abi lit y,str ing 65, 73
implementation wit h slice,stack 92,
215
implicit
& 158, 167
assig nment38
conv ersion79
dereference 159
import de clarat ion3,28, 42, 284,
285, 362
import
bl ank 287
path 284
path dom ain name284
renaming286
incr func tion33
incrementstatement ++ 5, 37, 94
index operat ion, str ing 64
indirec tion operator * 24, 32
infinite loop6,120, 228
infor mat ionhiding168, 284
init func tion44, 49
initializat ion
lazy 268
package44
st atement in if 22, 206
st atement in switch 24
initializer list 30
injec tion att ack,HTML115
injec tion att ack,SQL 211
in-place slice techniques 91
insertionsor t algor it hm101
int type 52
integer
literal 55
ov erflow 53, 113
sig ned 52, 54
unsig ned 52, 54
integrat iontest314
interface
assig nabilit y 175
comp arison184
dy namic typ e 181
embedding174
error built-in 196
flag.Value 179, 180
fmt.Stringer 180, 210
http.Handler 191, 193
io.Closer 174
io.Reader 174
io.Writer 15, 22, 172, 174, 186,
208, 209, 309
JSON110
method cal l 182
ni l 182
pit fal l 184
ReadWriteCloser 174
ReadWriter 174
satisfac tion171, 175
sort.Interface 186
type 171, 174
interface{} type 143, 176, 331
interface
type ass ertion208, 210
type,emp ty176
value 181
with nil point er184
zerovalue 182
interfaces as cont rac ts 171
internal package298
intset example166
intsToString func tion74
invar iants 159, 169, 170, 265, 284,
311, 352
io package174
io.Closer interface 174
io.Copy func tion17, 18
io.Discard st ream22
io.Discard var iable 18
io.EOF var iable 132
io/ioutil package16, 145
io.Reader interface 174
iota cons tantgenerator xiii,77
ioutil.ReadAll func tion16, 272
ioutil.ReadDir func tion247
ioutil.ReadFile func tion12, 145
io.Writer interface 15, 22, 172,
174, 186, 208, 209, 309
io.WriteString func tion209
‘‘is a’’ re lat ions hip 162, 175
issue, Go110, 112, 358
issuetracker, GitHu b 110
issues example112
issueshtml example115
issuesreport example114
it erat ionorder,map 95
it erat ionvar iable,capturing 140
itob func tion64
Java programminglangu agexv
JavaS crip t Objec t No tat ion(JSON)
107, 338
JavaS crip t prog ramminglangu age
xv,107
jpeg example287
JSON
interface 110
interface,OpenMov ieDat abas e
113
interface,xkcd113
mars haling108
unmarsh aling110
json.Decoder type 111
json.Encoder type 111
json.Marshal func tion108
json.MarshalIndent func tion108
json.NewDecoder func tion111
json.NewEncoder func tion111
json.Unmarshal func tion110, 114
ke yword , type 212
ke yword s,table of27
Knut h,Don ald 323
labelscope 46
label, statement 46
labele d
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
374 INDEX
break st atement 249
continue st atement 249
st atement 46
layout,memor y 354, 355
lazy initializat ion268
le ak, goro utine 233, 236, 246
lef t shif t op erator << 54
len built-in function4,54, 64, 65,
81, 84, 233
lexic al block46, 120, 135, 141, 212
lexic al closure136
lifet ime, var iable 35, 46, 135
links examplepackage138
links.Extract func tion138
Lisp programminglangu age338
Liss ajous algor it hm15
Liss ajous figure13
lissajous example14, 22, 35
list,initializer 30
literal
ar ray 82, 84
comp osite 14
func tion22, 135, 227
hexade cimal 55
imag inar y 61
integer55
map94
octal55
raw str ing 66
rune 56
slice 38, 86
st ring65
st ruc t 15, 102, 106
lo cal
block46
var iable 29, 141
var iable,addressof32, 36
var iable scope 135
lo cat ingpackages 291
lo ck
cont ent ion 267, 272
exclusive263, 266, 270
mu tex 102, 263, 264, 324
non-reent rant265
re aders266
sh are d 266
wr iter266
log package49, 130, 170
log.Fatalf func tion49, 130
lo oku p m[key],map 94
lo oku p, ok value fro m map96
lo op
infinite 6,120, 228
range 6, 9
var iable,capturing 141, 236, 240
var iable scope 141, 236
while6
main func tion2,310
main,package2,285, 310
make built-in function9,18, 88, 94,
225
make ch annel 18, 225
make map9,18, 94
make slice 88, 322
Mandelbro t figure63
Mandelbro t set61
mandelbrot example62
map
as set 96, 202
comp arison96
element, non existent94, 95
it erat ionorder 95
literal 94
lo oku p m[key] 94
lo oku p, ok value fro m 96
make 9, 18, 94
ni l 95
range ov er94
type 9,93
with slice key 97
zerovalue 95
mars halingJSON108
math package14, 56
math/big package63
math/cmplx package61
math.Hypot func tion156
math.Inf func tion57
math.IsInf func tion57
math.IsNaN func tion57
math.NaN func tion57
math/rand package285, 308
memo example275, 276, 277, 278,
279
memo examplepackage273
memoizat ion272
memory allocat ion36, 71, 89, 169,
209, 322
memory layout354, 355
metach arac ter,HTML116
method
(*bufio.Reader).ReadRune 98
(*bufio.Scanner).Err 97
(*bufio.Scanner).Scan 9
(*bufio.Scanner).Split 99
(*bytes.Buffer).Grow 169
(*bytes.Buffer).WriteByte 74
(*bytes.Buffer).WriteRune 74
(*bytes.Buffer).WriteString
74
call,int erface 182
ch aining114
de clarat ion40, 155
error.Error 196
Expr.Check 202
expression164
Expr.Eval 199
geometry.Point.Distance 156
gett er169
(*http.Client).Do 253
(*http.Request).ParseForm 22,
348
name 156
net.Conn.Close 220
net.Listener.Accept 220
(*os.File).Write 183
path.Distance 157
prom otion 161
re ceivername157
re ceiverparameter 156
re ceivertyp e 157
reflect.Type.Field 348
reflect.Value.Addr 342
reflect.Value.CanAddr 342
reflect.Value.Interface 331,
342
reflect.Value.Kind 332
sele ctor156
sett er169
String 40, 166, 329
(*sync.Mutex).Lock 21, 146, 263
(*sync.Mutex).Unlock 21, 146,
263
(*sync.Once).Do 270
(*sync.RWMutex).RLock 266
(*sync.RWMutex).RUnlock 266
(*sync.WaitGroup).Add 238
(*sync.WaitGroup).Done 238
template.Funcs 114
template.Parse 114
(*testing.T).Errorf 200, 304,
306
(*testing.T).Fatal 306
time.Time.Format 220
value 164
(*xml.Decoder).Token 213
methods example351
methodsofatyp e,displ aying 351
Micros oft Windows file name72
missingfunctionbody121
m[key],map lookup 94
mobi leplatfor ms121
Mo dula-2 programminglangu age
xiii
mo dular ity 283
monitor264, 275
monitorgoroutine 261, 277
movie example108, 110
moving garb agecol lec tor 357
mu ltimap160, 193
mu ltiple-value assig nment37
mu ltiplexer,HTTPrequest193
mu ltiplexing, event 244
mu ltiplexing, goroutine 281
mu ltithre ading, share d-memor y
217, 257
mu lti-value d func tion11, 30, 37, 96,
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
INDEX 375
125, 126
mustCopy func tion221
mu tex 145, 163, 256, 269
gu arding263
lo ck102, 263, 264, 324
re ad/w rit e 266, 267
mu tualexc lusion262, 267
MySQLdat abas e dr iver284
name
method 156
method receiver157
package28, 43
parameter 120
sp ace 41, 156, 283
name d
resu lt120, 126
resu ltzerovalue 120, 127
type 24, 39, 40, 105, 157
namingconvent ion 28, 169, 174,
289
naming, package289
NaN(notanumber) 57, 93
narrow ing conversion40, 55
negat ionoperator ! 63
net package219
netcat example221, 223, 227
net.Conn type 220
net.Conn.Close method 220
net.Dial func tion220
netflag example78
net/http package16, 191
net.Listen func tion220
net.Listener type 220
net.Listener.Accept method 220
net/smtp package312
net/url package160
networ king121, 219
new built-in function34
new,redefining35
ni l
ch annel 246, 249
func tion132
interface 182
map95
pointer32
pointer, int erface wit h 184
re ceiver159, 185
slice 87
non-blo cking
cach e 275
cach e, conc urrent272
ch annel receive 246
select 246
nonempty example92
nonexistentmap element94, 95
non-reent rantlock265
non-stand ard package121
numb er, floating-p oint56
numb erzerovalue 5, 30
numeric
conv ersion79
precision55, 78
type 51
Ob ero n prog ramminglangu agexiii
objec t 156
objec t-oriente d prog ramming
(O OP) 155, 168
octalescap e 66
octallit eral 55
ok value 37
ok value fro m ch annel receive 229
ok value fro m func tioncal l 128
ok value fro m maplooku p 96
ok value fro m type ass ertion206
omitempty eldtag 109
Op enMov ieDat abas e JSON
interface 113
op erat ion, atomic 264
op erat ion, conv ersion40, 55, 64, 71,
78, 79, 173, 187, 194, 208, 231,
353, 358
op erator
+=, -=,etc., assig nment5
&,address-of24, 32, 94, 158, 167
&^,AND-NOT 53
&^,bit-cle ar 53
^,bit w ise complement53
|,bit w ise OR166, 167
==,comparison40, 63
^,exc lusiveOR53
*,indirec tion 24, 32
<<,lef t shif t 54
!,negat ion63
%,remainder 52, 166
>>,rig htshif t 54
&&,short-circuit AND 63
||,short-circuit OR63
+,str ing con catenat ion5,65
-,unary 53
+,unary 53
asso ciativity 52
precedence 52, 63
s[i:j],slice 84, 86
s[i:j],subst ring65, 86
op erator s
assig nment36, 52
comp arison40, 93
tableofbinar y 52
tableofbit w ise 53
tableofcomparison53
opt imizat ion264, 321, 323
opt imizat ion, premature 324
OR operator ||,short-circuit 63
order ofdeclarat ions 48
order,str uct eld101, 355
or ganizat ion, wor ksp ace 291
OS threadvs. goroutine 280
os package4,206
os.Args var iable 4
os.Close func tion11
os.Exit func tion16, 34, 48
*os.File type 11, 13, 172, 175, 185,
336
os.FileInfo type 247
(*os.File).Write method 183
os.Getwd func tion48
os.IsExist func tion207
os.IsNotExist func tion207
os.IsPermission func tion207
os.LinkError type 207
os.Open func tion11
os.PathError type 207
os.Stat func tion247
outline example123, 133
// Output comment326
ov erflow,int eger53, 113
ov erflow,stack 124
package de clarat ion2,28, 41, 285
package
API284, 296, 311, 333, 352
back-do or315
bank example258, 261, 263
block46
bufio 9
bytes 71, 73
bzip example363
compress/bzip2 361
crypto/sha256 83
database/sql 211, 288
do cumentation 296
encoding/json 107
encoding/xml 107, 213
errors 196
exter nal test285, 314
flag 33, 179
fmt 2
format example332
geometry example156
golang.org/x/net/html 122
html/template 113, 115
http example192
image 62, 287
image/color 14
image/png 288
initializat ion44
internal 298
io 174
io/ioutil 16, 145
links example138
log 49, 130, 170
main 2, 285, 310
math 14, 56
math/big 63
math/cmplx 61
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
376 INDEX
math/rand 285, 308
memo example273
name 28, 43
naming289
net 219
net/http 16, 191
net/smtp 312
net/url 160
non-stand ard 121
os 4, 206
params example348
path 72
path/filepath 72
reflect 330
regexp 149
runtime 151
sort 95, 186, 189
storage example312, 313
strconv 22, 71, 75
strings 7, 71, 72, 289
sync 237, 263
syscall 196, 208
tempconv example42
testing 285, 302
text/scanner 344
text/tabwriter 188
text/template 113, 300
thumbnail example235
time 18, 77, 183
unicode 71
unicode/utf8 69
unsafe 354
package-le vel declarat ion28
packages
building293
down loading292
lo cat ing291
quer ying298
palindrom e 191
palindrom e example303, 305, 308
panic 64, 152, 253
panic built-in function148, 149
paradoxic al race 267
paral lelcomposition 224
paral lel, emb arrassingly235
paral lelism 217
parameter
... 91, 142, 143, 172
func tion119
method receiver156
name 120
passing120
unus ed120
params example348
params examplepackage348
params.Unpack func tion349
parent hes es 4, 6, 9, 52, 63, 119, 146,
158, 285, 335, 345
Parse example152
pars er, HTML121
Pascal programminglangu agexiii
path, ... 292, 299
path package72
path.Distance method 157
path/filepath package72
Pi ke,Rob xi, xiii,67, 107
pipeline example228, 230, 231
pip eline227
pip elinediagram 228
pit fal l,int erface 184
pit fal l,scope 140
pl atfor ms, mobile121
Pl ayg round,Goxvi,326
pl aylistexample187
png.Encode func tion62
pointer24, 32, 34
ali asing33
argument33, 83
equality 32
hidden 357
ni l 32
re ceiver158, 167
to str uct 100, 103
zerovalue 32
pollingchannel 246
poly mor phism, ad hoc 211
poly mor phism, subtype 211
PopCount func tion45
Po rtableNet wor k Graphics (PNG)
62
POSIX file name72
POSIX stand ard xi, 55, 72, 197
precedence,operator 52, 63
precision
floating-p oint56, 57, 63, 78
numeric55, 78
of con stants78
predeclare d names, table of28
prematureabstrac tion216, 316, 317
prematureopt imizat ion324
Printf %% 10
Printf verbs, table of10
Printf %b 10, 54, 75
Printf %c 10, 56
Printf %d 10, 55
Printf %e 10, 57
Printf %f 10, 57
Printf %g 10, 57
Printf %[n] 56
Printf %o 10, 55
Printf %q 10, 56, 97
Printf %s 10
Printf %*s 134
Printf %T 10, 80, 83, 184, 331
Printf %t 10, 83
Printf %#v 106, 207
Printf %v 10, 11
Printf % x 71
Printf %#x 56
Printf %x 10, 55, 83
produc tioncode301
profile
blocking 324
CPU 324
he ap324
profiling324
prog ramminglangu age
Alef xiii
APLxiii
C++ xiv, xv, 361
Cxii,xv, 1, 6, 52, 260, 361
Haskel l xiv
Java xv
JavaS crip t xv,107
Lisp 338
Mo dula-2 xiii
Ob ero n xiii
Pascal xiii
Python xv, 193
Ru byxv, 193
Scheme xiii
Sque ak, Ne wsque ak xiii
prom otion,met hod 161
prot ocolbuf fers107
Python prog ramminglangu agexv,
193
qu alified identifier 41, 43
quer yingpackages 298
quot e ch arac ter, ' 56
race
condit ion 21, 257, 258, 259
detec tor 271, 274
paradoxic al 267
randomize d test ing 307
range lo op6,9
range ov erchannel 229
range ov ermap 94
range ov erstr ing 69, 88
{{range}} temp lateaction113
raw str ing lit eral 66
re ach abi lit y 36
re ad, stale 268
re aderslock266
re ad/w rit e mu tex 266, 267
ReadWriteCloser interface 174
ReadWriter interface 174
real built-in function61
re ceive
<-ch,channel 18, 225, 232
expression225
non-blo cking channel 246
ok value fro m ch annel 229
re ceive-onlychannel typ e <-chan T
230
re ceiver
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
INDEX 377
name,met hod 157
ni l 159, 185
parameter,met hod 156
pointer158, 167
type,met hod 157
recover built-in function152
re cursion121, 124, 247, 333, 339,
345, 359
re cursive
anonymou s func tion137
data str ucture101, 102, 107
type 48
re defining new 35
reference
call by83
identity 87
type 9,12, 93, 120
reflect package330
reec tion 329, 352, 359
reflect.StructTag type 348
reflect.Type type 330
reflect.Type.Field method 348
reflect.TypeOf func tion330
reflect.Value type 331, 342
reflect.Value zerovalue 332
reflect.Value.Addr method 342
reflect.Value.CanAddr method
342
reflect.Value.Interface method
331, 342
reflect.Value.Kind method 332
reflect.ValueOf func tion331, 337
reflect.Zero func tion345
regexp package149
regexp.Compile func tion149
regexp.MustCompile func tion149
regu lar expression66, 149, 305, 321
re lat ion, ‘‘happ ens before’’ 226, 257,
261, 277
re lat ions hip, ‘‘hasa’’ 162
re lat ions hip, ‘‘is a’’ 162, 175
remainder operator % 52, 166
renamingimp ort 286
rendezvous 234
repl acementcharac ter (,Unico de
70, 98
repository, gopl.io xv i
re quest
HT TP GET21, 127, 272, 348
HT TP POST348
mu ltiplexer,HTTP193
resu ltlist, function119
resu lt, named 120, 126
return,bare126
return st atement 29, 120, 125
rev example86
reverb example223, 224
rightshif t op erator >> 54
Ru byprogramminglangu agexv,
193
rune literal 56
rune type 52, 67
rune slice tostr ing conversion71
rune tostr ing conversion71
runtime package151
runt ime API 324
runt ime sch edu ler 281
runtime.Stack func tion151
satisfac tion, interface 171, 175
Scal able Vec tor Graphics (SVG) 58
sche duler,runtime 281
Scheme programminglangu agexiii
scop e
de clarat ion45, 137
for 47
if-else 47
label46
lo cal variable135
lo opvar iable 141, 236
pit fal l 140
short var iable decl arat ion22, 48
switch 47
search algor it hm, bre adt h-firs t 139,
239
search algor it hm, depth-firs t 136
SearchIssues func tion111
select case 245
select, default case in246
select,non-blo cking 246
select st atement 244, 245
select{} st atement 245
sele ctive recov ery 152
sele ctor, eld156
sele ctor, met hod 156
semaph ore , binary 262
semaph ore , counting 241
semicolon3,6
send ch<-,channel 18, 225, 232
send statement 225
send-onlychannel typ e chan<- T
230
separatecompi lat ion284
sequence diag ram, thumbnail 238
sequential consistenc y 268, 269
serial confinement 262
server example19, 21
server
ch at253
conc urrentclo ck219
conc urrentech o 222
set, map as 96, 202
sett ermet hod 169
sexpr example340
S-expression
de code example347
de coding344
enco ding338
sexpr.Marshal func tion340
sexpr.readList func tion347
sexpr.Unmarshal func tion347
SHA256 message digest83
sha256 example83
sh adowing declarat ion46, 49, 206,
212
sh are d
lo ck266
var iables 257
var iables, conc urrency wit h 257
sh are d-memor y mu ltithre ading217,
257
shif t op erator <<,lef t 54
shif t op erator >>,rig ht54
short
var iable decl arat ion5, 7, 30, 31
var iable decl arat ionscope 22, 48
var iable decl arat ionstatement 7
short-circuit
AND operator && 63
evaluation 63
OR operator || 63
sig nature, function120
sig ned int eger52, 54
s[i:j],slice operator 84, 86
s[i:j],subst ringoperator 65, 86
simplestatement 6,22
Sizeof table354
sleep example179
slice 4
argument86
capacity 88, 89
capacity growt h di agram 90
comp arison87
extending a 86
ke y,map wit h 97
literal 38, 86
make 88, 322
ni l 87
of monthsdiagram 84
op erator s[i:j] 84, 86
ro tat ionalgor it hm86
te chniques, in-place 91
type 84
us ed as stack 123
zerolengt h 87
zerovalue 74, 87
SMTP client312
so cket
TCP219
UDP219
Unix domain 219
sort algor it hm, top olog ical 136
sort package95, 186, 189
sort.Float64s func tion191
sort.Interface interface 186
sort.Ints func tion191
sort.IntsAreSorted func tion191
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
378 INDEX
sort.IntSlice type 191
sort.Reverse func tion189
sort.Strings func tion95, 137, 191
spinner example218
Sprint func tion330
SQLAPI 211
SQLinj e ction att ack 211
sqlQuote func tion211, 212
squares example135
Sque ak, Ne wsque ak prog ramming
language xiii
st ack
al location 36
xe d-size 124
growth 124, 280, 358
implementation wit h slice 92, 215
ov erflow 124
slice usedas123
trace 149, 253
var iable 36
var iable-size 124
st ale read268
st and ard
IEEE 754 56, 57
POSIX xi,55, 72, 197
Unico de 2, 27, 52, 66, 67, 69, 97
st atement
--,decrement 5,37
++,increment 5,37, 94
assig nment5, 7, 36, 52, 94, 173
break 24, 46
continue 24, 46
coverage318, 320
defer 144, 150, 264
fallthrough 23, 212
for 6
go 18, 218
goto 24
if-else 9, 22, 47
label46
labele d 46
return 29, 120, 125
select{} 245
select 244, 245
send 225
short var iable decl arat ion7
simple6,22
switch 23, 47
tagless switch 24
type switch 210, 212, 214, 329
unreach able120
storage examplepackage312, 313
St range lov e,Dr. 336
st rateg ies, er ror-handling128, 152,
310, 316
strconv package22, 71, 75
strconv.Atoi func tion22, 75
strconv.FormatInt func tion75
strconv.Itoa func tion75
strconv.ParseInt func tion75
strconv.ParseUint func tion75
st ream, io.Discard 22
String method 40, 166, 329
st ring
conc atenation operator + 5, 65
conv ersion71
immut abi lit y 65, 73
index operat ion64
literal 65
literal,raw 66
range ov er69, 88
sh aring diagram 65
test 71
to byteslice conv ersion40, 73
to runeslice conv ersion71, 88
zerovalue 5, 7, 30
comp arison65
strings package7,71, 72, 289
strings.Contains func tion69
strings.HasPrefix func tion69
strings.HasSuffix func tion69
strings.Index func tion289
strings.Join func tion7,12
strings.Map func tion133
strings.NewReader func tion289
strings.NewReplacer func tion289
strings.Reader type 289
strings.Replacer type 289
strings.Split func tion12
strings.ToLower func tion72
strings.ToUpper func tion72
struct de clarat ion99
st ruc t
comp arison104
embedding104, 161
Employee 100
empt y 102
eld15, 99
eld, anony mou s 104, 105, 106,
162
eld, emb edde d 161
eld, exp ort of101, 106, 109, 110,
168
eldorder 101, 355
eldtag 109, 348
hole 354
hole diag ram 355
literal 15, 102, 106
literal,addressof103
pointerto100, 103
type 15, 24, 99
struct{} type 227, 241, 250
st ruc t type,unnamed 163
st ruc t zerovalue 102
su bst itutabi lit y 193
su bst ringoperator s[i:j] 65, 86
su btype poly mor phism 211
sum example142
surface example59, 203
surface figure, 3-D 58, 203
SVG58
SWIG361
Sw iss armyknife290
switch, default case in23
switch,initializat ionstatement in
24
switch scop e 47
switch st atement 23, 47
switch st atement,tag less24
switch st atement,typ e 210, 212,
214, 329
sw itch,context 280
sync package237, 263
sy nchro nou s ch annel 226
sync.Mutex type 263, 269
(*sync.Mutex).Lock method 21,
146, 263
(*sync.Mutex).Unlock method 21,
146, 263
sync.Once type 270
(*sync.Once).Do method 270
sync.RWMutex type 266, 270
(*sync.RWMutex).RLock method
266
(*sync.RWMutex).RUnlock method
266
sync.WaitGroup type 237, 250, 274
(*sync.WaitGroup).Add method
238
(*sync.WaitGroup).Done method
238
syscall package196, 208
syscall.Errno type 196, 197
systemcal l API196
tableof
binary operator s 52
bit w ise operator s 53
comp arisonoperator s 53
es cap e sequences 66
ke yword s 27
predeclare d names28
Printf verbs 10
UTF-8 enco dings 67
table, Sizeof 354
table-dr iventesting 200, 306, 319
tag, str uct eld109, 348
tagless switch st atement 24
tags, bui ld 296
TCPsocket219
te chniques, in-place slice 91
tempconv example39, 180, 289
tempconv examplepackage42
temp eratureconversionexample29
tempflag example181
template API115
temp late
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
INDEX 379
| in 113
ac tion, {{range}} 113
dot . in 113
template.Funcs method 114
template.HTML type 116
template.Must func tion114
template.New func tion114
template.Parse method 114
Test func tion302
test
bl ack-b ox310
br ittle317
ch arac ter 71
coverage318
dep endency,cyc lic 314
echo 309
fai luremessage 306
integrat ion314
of word example303
package, exter nal 285, 314
st ring71
whit e-b ox311
assertion306
_test.go file 285, 302, 303
testing package285, 302
test ing
acommand 308
randomize d 307
table-dr iven200, 306, 319
testing.B type 321
testing.T type 302
(*testing.T).Errorf method 200,
304, 306
(*testing.T).Fatal method 306
tests, writing effec tive 316, 317
text/scanner package344
text/tabwriter package188
text/template package113, 300
Thomps on, Ken xi,67
thre ad218, 280
thre ad-lo cal storage282
3-D sur face figure58, 203
thumbnail example236, 237, 238
thumbnail examplepackage235
thumbnail sequence diag ram 238
time package18, 77, 183
time.After func tion245
time.AfterFunc func tion164
time.Duration type 76, 179
time.Minute cons tant76
time.Now func tion220
time.Parse func tion220
time.Second cons tant164
time.Since func tion114
time.Tick func tion244, 246
time.Time type 114
time.Time.Format method 220
title example153
title func tion144, 145
token-b ased decoder API213, 215,
347
token-b ased XML deco ding213
to ol
cgo 361, 362
go 2, 42, 44, 290
go doc 25
godoc xv i,25, 297, 326
gofmt 3, 4, 44, 286
goimports 3, 44, 286
golint 292
topolog ical sor t algor it hm136
topoSort example136
trace example146
trace,stack 149, 253
tree,binar y 102
treesort example102
true boole an cons tant63
tr unc ation,floating-p oint40, 55
tupleassig nment31, 37
type de clarat ion39
type ke yword 212
type
abstrac t 24, 171
ag gregate81, 99
ar ray 81
assertion205, 211
assertion, interface 208, 210
assertion, ok value fro m 206
bool 63
bufio.Scanner 9
byte 52
bytes.Buffer 74, 169, 172, 185
Celsius 39
chan 225
ch annel 18
<-chan T,receive-onlychannel
230
chan<- T,send-onlychannel 230
complex 61
comp osite xv, 14, 81
comp osition xv, 107, 162, 189
concrete 24, 171, 211, 214
displaying met hodsof a 351
empt y interface 176
error built-in 11, 128, 149, 196
Fahrenheit 39
func tion119, 120
http.HandlerFunc 194, 203
http.Request 21, 253
http.ResponseWriter 19, 22,
191, 193
int 52
interface{} 143, 176, 331
interface 171, 174
interface dynamic 181
json.Decoder 111
json.Encoder 111
map9,93
method receiver157
mismatch 55
name d 24, 39, 40, 105, 157
net.Conn 220
net.Listener 220
numeric51
*os.File 11, 13, 172, 175, 185,
336
os.FileInfo 247
os.LinkError 207
os.PathError 207
re cursive 48
reference9,12, 93, 120
reflect.StructTag 348
reflect.Type 330
reflect.Value 331, 342
rune 52, 67
slice 84
sort.IntSlice 191
strings.Reader 289
strings.Replacer 289
struct{} 227, 241, 250
st ruc t 15, 24, 99
switch, case in 212
switch, default case in212
switch st atement 210, 212, 214,
329
sync.Mutex 263, 269
sync.Once 270
sync.RWMutex 266, 270
sync.WaitGroup 237, 250, 274
syscall.Errno 196, 197
template.HTML 116
testing.B 321
testing.T 302
time.Duration 76, 179
time.Time 114
uint 52
uintptr 52, 354, 357
underly ing 39
unidirec tion alchannel 230, 231
unnamed str uct 163
unsafe.Pointer 356
url.URL 193
types, untypedcon stant 78
UDPsocket219
uint type 52
uintptr type 52, 354, 357
unar y op erator + 53
unar y op erator - 53
unbuffered channel 226
undefine d behavior 260
underly ing array 84, 88, 91, 187
underly ing typ e 39
Unico de
co de point67
es cap e 68, 107
repl acementcharac ter ( 70, 98
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
380 INDEX
st and ard 2,27, 52, 66, 67, 69, 97
unicode package71
unicode.IsDigit func tion71
unicode.IsLetter func tion71
unicode.IsLower func tion71
unicode.IsSpace func tion93
unicode.IsUpper func tion71
unicode/utf8 package69
unidirec tion alchannel typ e 230, 231
union, discriminated 211, 213, 214
univers e block46
Unix domain socket219
unmarsh alingJSON110
unnamed str uct typ e 163
unnamed var iable 34, 88
unreach ablestatement 120
unsafe package354
unsafe.AlignOf func tion355
unsafe.Offsetof func tion355
unsafe.Pointer conv ersion356
unsafe.Pointer type 356
unsafe.Pointer zerovalue 356
unsafe.Sizeof func tion354
unsig ned int eger52, 54
untypedcon stant typ es 78
unus edparameter 120
URL 123
URL escape 111
url.QueryEscape func tion111
url.URL type 193
urlvalues example160
UTF-8 66, 67, 98
UTF-8 enco dings, table of67
utf8.DecodeRuneInString func tion
69
utf8.RuneCountInString func tion
69
utf8.UTFMax value 98
value
addressable32
call by83, 120, 158
func tion132
interface 181
method 164
utf8.UTFMax 98
var de clarat ion5,30
var iable
confinement 261
he ap36
http.DefaultClient 253
io.Discard 18
io.EOF 132
lifet ime35, 46, 135
lo cal 29, 141
os.Args 4
st ack 36
unnamed 34, 88
var iables, escaping 36
var iables, share d 257
var iable-size stack 124
var iadic function142, 172
ve ctor, bit 165
vendor ing 293
visibi lit y 28, 29, 41, 168, 297
visit func tion122
wait example130
WaitForServer func tion130
walkDir func tion247
we b
craw ler 119
craw ler,con cur rent239
framewor k 193
whileloop6
whit e-b oxtest311
Wi l kes, Maur ice 301
Wirth, Nik lausxiii
word example303, 305, 308
word example, test of303
worksp ace organizat ion291
wr iterlock266
wr iting effec tive tests 316, 317
xkcd JSONint erface 113
XML deco ding213
XML (ExtensibleMarku p Language)
107
(*xml.Decoder).Token method
213
xmlselect example215
zerolengt h slice 87
zerovalue
ar ray 82
boole an 30
ch annel 225, 246
func tion132
interface 182
map95
name d resu lt120, 127
numb er5,30
pointer32
reflect.Value 332
slice 74, 87
st ring5, 7, 30
st ruc t 102
unsafe.Pointer 356
The Go Programming Language
© 2016 Alan A. A. Donovan & Brian W. Kernighan
revision 3b600c, date 29 Sep 2015
From the Library of YIGUANG HU
ptg16105617
You love our titles and you love to
share them with your colleagues and friends...why
not earn some $$ doing it!
If you have a website, blog, or even a Facebook
page, you can start earning money by putting
InformIT links on your page.
Whenever a visitor clicks on these links and makes
a purchase on informit.com, you earn commissions*
on all sales!
Every sale you bring to our site will earn you a
commission. All you have to do is post the links to
the titles you want, as many as you want, and we’ll
take care of the rest.
APPLY AND GET STARTED!
It’s quick and easy to apply.
To learn more go to:
http://www.informit.com/affiliates/
*Valid for all books, eBooks and video sales at www.informit.com
JOIN THE
INFORMIT
AFFILIATE TEAM!
From the Library of YIGUANG HU